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meet pytfon 

Everyone loves lists 

You’re asking one question: “What makes Python different?” 

The short answer is: lots ofthings. The longer answers starts by stating that there’s 
lots that’s familiar, too. Python is a lot like any other general-purpose programming 
language, with statements, expressioris, operators, functions, modules, methods, 
and classes. AII the usual stuff, really. And then there’s the other stuff Python provides 
that makes the programmer’s life—y our life—that little bit easier. You’11 start your tour 
of Python by learning about lists. But, before getting to that, there’s another important 
question that needs answering... 
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sWing your code 

Modules of functions 

Reusable code is great, but a shareable module is better. 

By sharing your code as a Python module, you open up your code to the entire Python 
community...and it’s always good to share, isn’t it? In this chapter, you’ll learn how to 
create, install, and distribute your own shareable modules. You’ll then load your module 
onto Python’s Software sharing site on the Web, so that everyone can benefit from your 
work. Along the way, you’ll pick up a few new tricks relating to Python’s functions, too. 
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files and exceptions 

Dealing with errors 

lt’s simply not enough to process your list data in your code. 

You need to be able to get your data into your programs with ease, too. It’s no surprise 
then that Python makes reading data from files easy. Which is great, until you 
consider what can go wrong when interacting with data extemal to your programs... 
and there are lots of things waiting to trip you up! When bad stuff happens, you need a 
strategy for getting out of trouble, and one such strategy is to deal with any exceptional 
situations using Python’s exception handling mechanism showcased in this chapter. 
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Take a closer look at the data 7 7 
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.. .Or add another level of exception handling 98 

So, which approach is best? 99 

You’re done... except for one small thing 101 

Be specific with your exceptions 102 

Your Python Toolbox 103 
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persistence 

Saving data to files 

It is truly great to be able to process your file-based data. 

But what happens to your data when you’re done? Of course, it’s best to save your 
data to a disk file, which allows you to use it again at some later date and time. Taking 
your memory-based data and storing it to disk is what persistence is all about. Python 
supports all the usual tools for writing to files and also provides some cool facilities for 
efficiently storing Python data. 
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Files are left open after an exception! 114 
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Generic hle I/O with pickle is the way to go! 137 

Your Python Toolbox 138 
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coinprejiencHrig data 

Work that data! 

Data comes in all shapes and sizes, formats and encodings. 

To work effectively with your data, you often have to manipulate and transform it into a 
common format to allow for efficient Processing, sorting, and storage. In this chapter, 
you’11 explore Python goodies that help you work your data up into a sweat, allowing 
you to achieve data-munging greatness. 
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ciihtoiTI data objects 

Bundling code with data 

lt’s important to match your data structure choice to your data. 

And that choice can make a big difference to the complexity of your code. In Python, 
although really useful, lists and sets aren’t the only game in town. The Python dictionary 
lets you organize your data for speedy lookup by associating your data with names, not 
numbers. And when Python’s built-in data structures don’t quite cut it, the Python class 
statement lets you define your own. This chapter shows you how. 


Goach Kelly is back (with a new file format) 174 
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Define a class 190 
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The importance of self 192 
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Web deVelopmerrt 

Puttnng it all together 

Sooner or later, you’ll want to share your app with lots of people. 

You have many options for doing this. Pop your code on PyPI, send out lots of emails, put 
your code on a CD or USB, or simply install your app manually on the computers of those 
people who need it. Sounds like a lot of work...not to mention boring. Also, what happens 
when you produce the next best version of your code? What happens then? How do 
you manage the update? Let’s face it: it’s such a pain that you’ll think up really Creative 
excuses not to. Luckily, you don’t have to do any of this: just create a webapp instead. And, 
as this chapter demonstrates, using Python for web development is a breeze. 
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mobile app cleVelopment 

Small devices 

Putting your data on the Web opens up all types of possibilities. 

Not only can anyone from anywhere interact with your webapp, but they are increasingly 
doing so from a collectiori of diverse computing devices: PCs, laptops, tablets, palmtops, 
and even mobile phones. And it’s not just humans interacting with your webapp that 
you have to support and worry about: bots are small programs that can automate web 
interactions and typically want your data, not your human-friendly HTML. In this chapter, 
you exploit Python on Coach Kelly’s mobile phone to write an app that interacts with your 
webapp’s data. 
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m an age your data 

Handlong input 

The Web and your phone are not just great ways to display data. 

They are also great tools to for accepting input from your users. Of course, once your 
webapp accepts data, it needs to put it somewhere, and the choices you make when 
deciding what and where this “somewhere” is are often the difference between a webapp 
that’s easy to grow and extend and one that isn’t. In this chapter, you’ll extend your 
webapp to accept data from the Web (via a browser or from an Android phone), as well 
as look at and enhance your back-end data-management Services. 
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scallng ygur webapp 

Getting real 


The Web is a great place to hostyour app...until things get real. 

Sooner or later, you’ll hit the jackpot and your webapp will be wildly successful. When 
that happens, your webapp goes from a handful of hits a day to thousands, possibly ten 
of thousands, or even more. Will you be ready? Will your web server handle the load? 
How will you know? What will it cost? Who will pay? Can your data model scale to 
millions upon millions of data items without slowing to a crawll Getting a webapp up and 
running is easy with Python and now, thanks to Google App Engine, scaling a Python 
webapp is achievable, too. 
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dealing wkh complexit/ 

Data wrangling 

lt’s great when you can apply Python to a specific domain area. 

Whether it’s web development, database management, or mobile apps, Python helps 
you get the job done by not getting in the way of you coding your solution. And then 
there’s the other types of problems: the ones you can’t categorize or attach to a domain. 
Problems that are in themselves so unique you have to look at them in a different, highly 
specific way. Creating bespoke Software Solutions to these type of problems is an area 
where Python excels. In this, your final chapter, you’11 stretch your Python skills to the 
limit and solve problems along the way. 
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how to use this book 


Who is this book for? 

If you can answer “yes” to all of these: 


o 

o 

o 


Do you already know how to program in another 
programming language? 

Do you wish you had the know-how to program Python, 
add it to your list of tools, and make it do new things? 

Do you prefer actually doing things and applying the stuff 
you learn over listening to someone in a lecture rattle on 
for hours on end? 


this book is for you. 


Who should probably back away frow this book? 

If you can answer “yes” to any of these: 


o 

e 

o 


Do you already know most of what you need to know to 
program with Python? 

Are you looking for a reference book to Python, one that 
covers all the details in excruciating detail? 

Would you rather have your toenails pulled out by 15 
screaming monkeys than learn something new? Do you 
believe a Python book should cover everything and if it 
bores the reader to tears in the process then so much the 
better? 


this book is not for you. 
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We know what vou're thinking 


u 


u 


How can this be a serious Python book? 
What’s with all the graphics?” 

Can I actually leam it this way?” 


55 


& ihi,ks 


We know what your brain is thinking 

Your brain craves novelty. It’s always searching, scanning, waiting for something 
unusual. It was built that way, and it helps you stay alive. 

So what does your brain do with all the routine, ordinary, normal things 
you encounter? Everything it can to stop them from interfering with the 
brain’s real ]oh —recording things that matter. It doesn’t bother saving the 
boring things; they never make it past the “this is obviously not important’ 
filter. 


THIS is : 




How does your brain know what’s important? Suppose you’re out for a day 
hike and a tiger jumps in front of you, what happens inside your head and 
body? 

Neurons fire. Emotions crank up. Chemicals surge. 

And that’s how your brain knows... 

This must be important! Don’t forget it! 

But imagine you’re at horne, or in a library. It’s a safe, warm, tiger-free zone. 

You’re studymg. Gettmg ready for an exam. Or trymg to learn some tough VN£'*** 
technical topic your boss thinks will take a week, ten days at the most. 

Just one problem. Your brain’s trying to do you a big favor. It’s trying to 
make sure that this obviously non-important content doesn’t clutter up scarce 
resources. Resources that are better spent storing the really big things. 

Like tigers. Like the danger of fire. Like how you should never have 
posted those “party” photos on your Facebook page. And there’s no 
simple way to teli your brain, “Hey brain, thank you very much, but 
no matter how dull this book is, and how little I’m registering on the 
emotional Richter scale right now, I really do want you to keep this 
stuff around.” 
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Metacognition: thiwkmg about thmkiwg 

If you really want to learn, and you want to learn more quickly and more 
deeply, pay attention to how you pay attention. Think about how you think. 
Learn how you learn. 

Most of us did not take courses on metacognition or learning theory when we 
were growing up. We were expected to learn, but rarely taught to learn. 




The trick is to get your brain to see the new material you’re learning as 
Really Important. Grucial to your well-being. As important as a tiger. 
Otherwise, you’re in for a constant battle, with your brain doing its best to 
keep the new content from sticking. 


But we assume that if you’re holding this book, you really want to learn how 
to design user-friendly websites. And you probably don’t want to spend a lot 
of time. If you want to use what you read in this book, you need to remember 
what you read. And for that, you Ve got to understand it. To get the most from 
this book, or any book or learning experience, take responsibility for your brain. 
Your brain on this content. 


So just how DO you get your brain to treat 
programming like it was a hungry tiger? 


There 5 s the slow, tedious way, or the faster, more effective way. The 
slow way is about sheer repetition. You obviously know that you are able to learn 
and remember even the dullest of topics if you keep pounding the same thing into your 
brain. With enough repetition, your brain says, “This doesn’t feel important to him, but he 
keeps looking at the same thing over and over and over , so I suppose it must be.” 


The faster way is to do any thing that increases brain activity, especially different 
types of brain activity. The things on the previous page are a big part of the solution, 
and they 5 re all things that have been proven to help your brain work in your favor. For 
example, studies show that putting words within the pictures they describe (as opposed to 
somewhere else in the page, like a caption or in the body text) causes your brain to try to 
makes sense of how the words and picture relate, and this causes more neurons to fire. 
More neurons firing = more chances for your brain to get that this is something worth 
paying attention to, and possibly recording. 


A conversational style helps because people tend to pay more attention when they 
perceive that theyVe in a conversation, since they 5 re expected to follow along and hold up 
their end. The amazing thing is, your brain doesn’t necessarily care that the “conversation” 
is between you and a book! On the other hand, if the writing style is formal and dry, your 
brain perceives it the same way you experience being lectured to while sitting in a roomful 
of passive attendees. No need to stay awake. 


But pictures and conversational style are just the beginning... 
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Here'$ what WE did: 

We used pictures , because your brain is tuned for visuals, not text. As far as your brairfs 
concerned, a picture really is worth a thousand words. And when text and pictures work 
together, we embedded the text in the pictures because your brain works more effectively 
when the text is within the thing the text refers to, as opposed to in a caption or buried in the 
text somewhere. 

We used redundancy , saying the same thing in different ways and with different media types, 
and multiple senses , to increase the chance that the content gets coded into more than one area 
of your brain. 

We used concepts and pictures in unexpected ways because your brain is tuned for novelty, 
and we used pictures and ideas with at least some emotional content , because your brain 
is tuned to pay attention to the biochemistry of emotions. That which causes you to feel 
something is more likely to be remembered, even if that feeling is nothing more than a little 

humor , surprise , or interest. 

We used a personalized, conversational style , because your brain is tuned to pay more 
attention when it believes you’re in a conversation than if it thinks you’re passively listening 
to a presentation. Your brain does this even when you’re reading. 

We included more than 80 activities , because your brain is tuned to learn and remember 
more when you do things than when you read about things. And we made the exercises 
challenging-yet-do-able, because that’s what most people prefer. 

We used multiple learning styles , because you might prefer step-by-step procedures, while 
someone else wants to understand the big picture first, and someone else just wants to see 
an example. But regardless of your own learning preference, everyone benefits from seeing the 
same content represented in multiple ways. 

We include content for hoth sides of your brain , because the more of your brain you 
engage, the more likely you are to learn and remember, and the longer you can stay focused. 
Since working one side of the brain often means giving the other side a chance to rest, you 
can be more productive at learning for a longer period of time. 

And we included stories and exercises that present more than one point of viezv, 
because your brain is tuned to learn more deeply when it 5 s forced to make evaluations and 
judgments. 

We included challenges , with exercises, and by asking questions that don’t always have 
a straight answer, because your brain is tuned to learn and remember when it has to work at 
something. Think about it—you can’t get your body in shape just by watching people at the 
gym. But we did our best to make sure that when you’re working hard, it 5 s on the right things. 
That you We not spending one extra dendrite processing a hard-to-understand example, 
or parsing difficult, jargon-laden, or overly terse text. 

We used people. In stories, examples, pictures, etc., because, well, because youPe a person. 
And your brain pays more attention to people than it does to things. 
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H ere'$ what YOU caw do to bewd 
your braiw into submission 

So, we did our part. The rest is up to you. These tips are a 
starting point; listen to your brain and figure out what works 
for you and what doesfft. Try new things. 

Cut this out 5hd s-fci£k it 

° h y° u ' r metiri gevaW. 


O 


o 


o 


o 


o 


o 


o 


Slow down. The more you understand, the o 
less you have to memorize. 

Dofft just read. Stop and think. When the book asks 
you a question, dofft just skip to the answer. Imagine 
that someone really is asking the question. The 
more deeply you force your brain to think, the better 
chance you have of learning and remembering. 

Do the exercises. Write your own notes. 

We put them in, but if we did them for you, that 
would be like having someone else do your workouts 
for you. And dofft just look at the exercises. Use a 
pencil . There’s plenty of evidence that physical 
activity while learning can increase the learning. 

Read the “There are No Dumb Questions.” 

That means all of them. They 5 re not optional 
sidebars, they We part of the core content! 

Dofft skip them. 

Make this the last thing you read before bed. o 
Or at least the last challenging thing. 

Part of the learning (especially the transfer to 
long-term memory) happens affer you put the book 
down. Your brain needs time on its own, to do more 
processing. If you put in something new during that 
Processing time, some of what you just learned will 
be lost. 

Talk about it. Out loud. 

Speaking activates a different part of the brain. If 
yoffre trying to understand something, or increase 
your chance of remembering it later, say it out loud. 

Better stili, try to explain it out loud to someone else. 

Yoffll learn more quickly, and you might uncover 
ideas you hadfft known were there when you were 
reading about it. 


Drink water. Lots of it. 

Your brain works best in a nice bath of fluid. 
Dehydration (which can happen before you ever 
feel thirsty) decreases cognitive function. 

Listen to your brain. 

Pay attention to whether your brain is getting 
overloaded. If you find yourself starting to skim 
the surface or forget what you just read, it’s time 
for a break. Once you go past a certain point, you 
wofft learn faster by trying to shove more in, and 
you might even hurt the process. 

Feel something. 

Your brain needs to know that this matters. Get 
involved with the stories. Make up your own 
captions for the photos. Groaning over a bad joke 
is stili better than feeling nothing at all. 

Write a lot of code! 

There’s only one way to learn to program: writing 
a lot of code. And that 5 s what yoffre going to 
do throughout this book. Goding is a skill, and the 
only way to get good at it is to practice. We’re going 
to give you a lot of practice: every chapter has 
exercises that pose a problem for you to solve. Dofft 
just skip over them—a lot of the learning happens 
when you solve the exercises. We included a solution 
to each exercise—dofft be afraid to peek at the 
solution if you get stuck! (It’s easy to get snagged 
on something small.) But try to solve the problem 
before you look at the solution. And definitely get it 
working before you move on to the next part of the 
book. 
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Read Me 

This is a learning experience, not a reference book. We deliberately stripped out everything 
that might get in the way of learning whatever it is we’re working on at that point in the 
book. And the first time through, you need to begin at the beginning, because the book 
makes assumptions about what you Ve already seen and learned. 

This book is designed to get you up to speed with Python as 
quickly as possible. 

As you need to know stuff, we teach it. So you won’t find long lists of technical material, no 
tables of Python’s operators, not its operator precedence rules. We don’t cover everything , 
but weVe worked really hard to cover the essential material as well as we can, so that you 
can get Python into your brain quickly and have it stay there. The only assumption we make 
is that you already know how to program in some other programming language. 

This book targets Python 3 

We use Release 3 of the Python programming language in this book, and we cover how to 
get and install Python 3 in the first chapter. That said, we don’t completely ignore Release 
2, as you’11 discover in Ghapters 8 through 11. But trust us, by then you’11 be so happy using 
Python, you won’t notice that the technologies you 5 re programming are running Python 2. 


We put Python to work for you right away. 

We get you doing useful stuff in Chapter 1 and build from there. There’s no hanging 
around, because we want you to be productive with Python right away. 

The activities are NOT optional. 

The exercises and activities are not add-ons; theyVe part of the core content of the book. 
Some of them are to help with memory, some are for understanding, and some will help 
you apply what you Ve learned. Don’t skip the exercises. 

The redundancy is intentional and important. 

One distinet difference in a Head First book is that we want you to really get it. And we 
want you to finish the book remembering what you Ve learned. Most reference books don’t 
have retention and recall as a goal, but this book is about learning , so youdl see some of the 
same concepts come up more than once. 


xxx intro 


The examples are as lean as possible. 

Our readers teli us that it’s frustrating to wade through 200 lines of an example looking 
for the two lines they need to understand. Most examples in this book are shown within 
the smallest possible context, so that the part you’re trying to learn is ciear and simple. 
Don’t expect all of the examples to be robust, or even complete—they are written 
specifically for learning, and aren’t always fully functional. 

WeVe placed a lot of the code examples on the Web so you can copy and paste them as 
needed. Youdl find them at two locations: 


http://www.headfirstlabs.com/books/hfpython/ 
http://python, itcarlow. ie 

The Brain Power exercises don’t have answers. 

For some of them, there is no right answer, and for others, part of the learning 
experience of the Brain Power activities is for you to decide if and when your answers 
are right. In some of the Brain Power exercises, you will find hints to point you in the 
right direction. 
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The technical review team 



Technical Reviewers: 


David Griffiths is the author of Head First Rails 
and the coauthor of Head First Programming. He began 
programming at age 12, when he saw a documentary 
on the work of Seymour Papert. At age 15, he wrote 
an implementation of Papert’s computer language 
LOGO. After studying Pure Mathematics at University, 
he began writing code for computers and magazine 
articles for humans. He’s worked as an agile coach, 
a developer, and a garage attendant, but not in that 
order. He can write code in over 10 languages and 
prose in just one, and when not writing, coding, or 
coaching, he spends much of his spare time traveling 
with his lovely wife—and fellow Head First author— 
Dawn. 


Phil Hartley has a degree in Computer Science 
from Edinburgh, Scotland. Having spent more than 
30 years in the IT industry with specific expertise in 
OOP, he is now teaching full time at the University 
of Advancing Technology in Tempe, AZ. In his spare 
time, Phil is a raving NFL fanatic 

Jeremy Jones is coauthor of Python for Unix and 
Linux System Administration. He has been actively using 
Python since 2001. He has been a developer, system 
administrator, quality assurance engineer, and tech 
support analyst. They all have their rewards and 
challenges, but his most challenging and rewarding job 
has been husband and father. 
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books and videos to find the answers you need quickly. 


With a subscription, you can read any page and watch any video from our library 
online. Read books on your cell phone and mobile devices. Access new tities before 
they are available for print, and get exclusive access to manuscripts in development 
and post feedback for the authors. Gopy and paste code samples, organize your 
favorites, download chapters, bookmark key sections, create notes, print out pages, 
and benefit from tons of other time-saving features. 


0’Reilly Media has uploaded this book to the Safari Books Online Service. To have 
full digital access to this book and others on similar topics from 0’Reilly and other 
publishers, sign up for free at http:/'/my.safaribooksonline.com/?portal—oreilly. 
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intro 


1 meet python 


* Everyone loves lists + 



You’re asking one question: “What makes Python different?” 


The short answer is: lots ofthings. The longer answers starts by stating that there’s lots 
that’s familiar, too. Python is a lot like any other general-purpose programming language, 
with statements, expressioris, operators, functions, modules, methods, and classes. 
AII the usual stuff, really. And then there’s the other stuff Python provides that makes 
the programmer’s life—y our life—that little bit easier. You’11 start your tour of Python by 
learning about lists. But, before getting to that, there’s another important question that 
needs answering... 


this is a new chapter 





python greatness 


Whaf $ to like about Python? 

Lots. Rather than teli you, this book 5 s goal is to show you the greatness that is 
Python. 




Yeah... I need something that I can deploy 
on PCs, Macs, handhelds, phones,the Web, 
on big servers and small clients...and it has 
to let me build GUIs quickly and painlessly., 
OK, yes, yeah, I'm listening... What?!? 
YouVe kidding! Python can do all that? 



Before diving head first into Python, let 5 s get a bit of housekeeping out of 
the way. 

To work with and execute the Python code in this book, you need a copy of 
the Python 3 interpreter on your computer. Like a lot of things to do with 
Python, it 5 s not difficult to install the interpreter. Assuming, of course, it 5 s not 
already the re... 


2 Chapter 1 


meet python 


Iwstall Python 3 


Before you write and run Python code, you need to make sure the Python 
interpreter is on your computer. In this book, you’11 start out with Release 3 of 
Python, the very latest (and best) version of the language. 

A release of Python might already be on your computer. Mac OS X comes 
with Python 2 preinstalled, as do most versions of Linux (which can also ship 
with Release 3). Windows, in contrast, doesn’t include any release of Python. 
Let’s check your computer for Python 3. Open up a command-line prompt 
and, if you are using Mac OS X or Linux, type: 


python3 -V 

On Windows, use this command: 

c: \Python31\python . 




« n 
V 


exe -V 


wpbrcass 

b y the way. 





V 


Usro<\ the 

UppfcRCASt 

«sdt* m ^e P^hon 

vcrsion ayyeavm^ 


If Python 3 is missing from 
your computer, download 
a copy for your favo rite OS 
from the zvwzv.python.org 
website. 


sCreen- 


u w 


lAAthout the 
UPPBRCASE v 
you ave take» 
m-to the Python 
mtevyvetev 



I File Edit Window Help WhichPython? 


$ python3 -V 
Python 3.1.2 
$ 

$ python3 

Python 3.1.2 (r312:79360M, Mar 24 2010, 01:33:18) 

[GCC 4.0.1 (Apple Inc. build 5493)] on darwin 

Type "help", "Copyright", "credits" or "license" for more info. 
»> 

»> quit() 

$ 


Use “the <\urtO 
£oww>3nd “to 
-the m-tcvyvftcir and 
«W -to 7 <>uv 0£ 


When you install Python 3, you also get IDLE, Python’s simple—yet 
surprisingly useful — integrated development environment. IDLE includes a 
color syntax-highlighting editor, a debugger, the Python Shell, and a complete 
copy of Python 3’s Online documentation set. 

Let’s take a quick look at IDLE. 
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idle hands 


Use IPU to help learn Python 


IDLE lets you write code in its full-featured code editor as well as experiment 
with code at the Python Shell. You’11 use the code editor later in this book 
but, when learning Python, IDLEs shell really rocks, because it lets you try 
out new Python code as you go. 


When you first start IDLE, you are presented with the “triple chevron” 
prompt (»>) at which you enter code. The shell takes your code statement 
and immediately executes it for you, displaying any results produced on screen. 


IDLE knows all about Python syntax and offers “completion hints” that pop 
up when you use a built-in function like print () . Python programmers 
generally refer to built-in functions as BIFs. The print () BIF displays 
messages to Standard output (usually the screen). 

tr>tev v/owv t-ode 
at tfce »> 


Wnlike othev C-based 
la«guages, wbidb use { and } 
to delimit blodks, pythoh uses 
ihdch-ta-tioh ihstcad. 



* O O 


Python ShelE 


r 

Scc v-csults 

^cdiatcly. ^ 


inrt 


Python 3.1.2 ( r312:7936QM, Mar 24 2010, 01:33;1B) 

[ GCC 4.0.1 (Apple Inc. build 5493)] on darwin 

Type "Copyright 1 ', ''credits" or Ticense( ) " for more Information. 

>» 

»> print ( "You can experiment with code within ( s shell, Cool, eh? 1 ") 

You can experiment with code within IDLE F s ^k^il . Cool, eh? 

»> if 43 > 42: 

print ( 11 Don r t panici 11 ) 


Don f t panici 
»> I 


Ln: 12 Coi: 4 


A 


IDLE uses colored syntax to highlight your code. By default, built-in 
functions are purple, strings are green, and language keywords (like if) are 
orange. Any results produced are in blue. If you hate these color choices, 
doni worry; you can easily change them by adjusting IDLE’s preferences. 

IDLE also knows all about Pythoni indentation syntax, which requires code 
blocks be indented. When you start with Python, this can be hard to get 
used to, but IDLE keeps you straight by automatically indenting as needed. 


IDLE knows 

Pythoni syntax 
and helps you 
confornt to 
the Python 
indentation rutes. 
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meet python 


Work effectively with IPLE 


IDLE has lots of features, but you need to know about only 
a few of them to get going. 

TAP cowpletiow 

Start to type in some code, and then press the TAB key. 
IDLE will offer suggestions to help you complete your 
statement. 


This is how IDLB looks oh 

I 7 ii° 7 ci e " P ^ hilooka 

l'ttle di+Wh-t oh youvs, but 
»°t by «,udb. (Ahd, y es , iCs 
ixeaht to look tbis ugly.) 


JyyC dr\d 

TA3 a-t >» Y* 0 ™ 

-to see IPL^s o-f 

su^Cstioy\S. 




« ^ n 


*Python Shell* 


Pyth 


»> 

»> 

You 

»> 


Don ’ 
>» pr[ 


_ a _a _a_ i i n_ =l- 

print 

| 

property 


quit 


range 


repr 


reversed 

0, 

round 

set 

▲ 

setattr 

▼ 

slice 



i60M, Mar 24 2010, 01:33:18) 
build 5493)) on darwin 


-'lanie 1 " ) 


Ln: 12 Coi: 6 


Recall code statewewts 


Press Alt-P to recall the previous code statement entered into 
IDLE or press Alt-N to move to the next code statement 
(assuming there is one). Both key combinations can be used 
to cycle rapidly through all of the code you Ve entered into 
IDLE, re-executing any code statements as needed. 


Alt-P for Previous 
Alt-N for Next 



Uttless youVe o* J 
a Mad, m whidh 

dasc vbV CW-P 

a Y\d Ctvl-K* 


Edit recalled code 

Once you recall your code statement, you can edit it and 
move around the statement using the arrow keys. It 5 s 
possible to edit any statement that youVe previously 
entered, even code statements that span multiple lines. 

Adjust IPIPs preferences 

IDLE’s preferences dialog lets you adjust its default Tw®ak |PL-£- 

behavior to your tastes. There are four tabs of settings to ^ y ouV - he3V"tV —^ 

tweak. You can control font and tab behavior, the colors £ 0 yrlewt- 

used to syntax highlight, the behavior of certain key- 

combinations, and IDLE’s start-up settings. So, if shocking 

pink strings is really your thing, IDLE gives you the power 

to change how your code looks on screen. 



IDL£ Preferences 


Fonts/Tabs Highlightmg Keys General 


Custom Highlighting 


Chouse Colour for : 


Normal Tcxt 


© Foreground .J. R.irkgrr>unri 


#you can elicit here 
etu ehviw» it cimi 

ao( func(paraa): 

verO " *etrinq' 

vari • aoloct cd 

var2 ■ CE ZM 

var3 • 1 tet.(None) 

errori cursor [ 

■hei 1 nrnnut «Mrrr 


Cive .is New Cusiom Theme 


Highlightmg Theme 
Select: 

(•) .i fttjilr ln Theme 

IDl F Classir 


no custom themcs 

V 

r D«iete Custom Therr 

vt 


Ok 


Apply 


Cancel 


Help 


you are here ► 
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dealing with data 


Peal with complex data 


Any program of any worth that you create has to work with data. Sometimes, 
the data is simple and straightforward —easy to work with. Other times, the 
data you have to work with is complex in its structure and meaning, forcing you 
to work hard to make sense of it ali, let alone write code to process it. 

To tame complexity, you can often arrange your data as a list: there’s the list 
of customers, your friend 5 s list, the shopping list, and your to-do list (to name 
a few). Arranging data in lists is so common that Python makes it easy for you 
to create and process lists in code. 


Let’s look at some complex data before learning how to create and process list 
data with Python. 


A 

w ov'te V>uW 

- • i . • ... 

• •. e 'xTi- a» •• « 1 «.• -• 



'rJifX: 

y/jf. 


The Holy djrail, 1775, Terry Jo»es i Terry Millia», II ■*» 

<jrahai» Chap»a« 

Miehael Pali», John Cleese, Terry ^ill»». Erit Idle i Terry Jones 
The Life of Bria», IT71. Terry Jo»es, 1+ «■'■»* 

^raham Chap>»a» 

Aliehael Pali», Joh» Cleese, Terry $lli*», Erit Idle f Terry Jonei 





The si* Monty Pytho» east »ie«ibers 


There 

sure is 
3 lo-fc o-P da-fca 
lis-ted here. 



The s,* Mo»ty Pytho» east ~e»bo-i 

The Holy $rail, 1175, Terry Jo»es ! Terr 

$illia», V *»i»s 

^raha» Chap,»a» 

Mithael Pali», Joh» Cleese. Terry ^illia» 

Erit Idle f Terry Jo»es 

The Life of Bria», If71, Terry Jones, H »•>»» 

^raha» Chapr-a» 

Mithael Pali», joh» Cleese, Terry ^illia», 

Erit Idle { Terry Jo»es 



^This data * to^le*, W 


On first glance, this collection of data does indeed look quite complex. 
However, the data appears to conform to some sort of structure: there 5 s a line 
for a list of basic movie facts, then another line for the lead actor(s), followed 
by a third line listing the movie’s supporting actors. 


This looks like a structure you can work with... 
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meet python 


Create simple Python lists 

Let’s start with the following simple list of movie tities and work up from 
there: 

The ttoly £jrail 
The Li-fe o-f Bria* 


The Meanmg ot Lite ^ ^4. |- is t 

Mcm-ty Pyt* 

Here’s the same list written in a way that Python understands: 

i 

o-f sor*e 

\oy\ movies 

> 


movies = ["The . 

"The 

"The 1 

Holy Grail", 

Life of Brian", 

Meaning of Life"] 


To turn the human-friendly list into a Python-friendly one, follow this four- 
step process: 


o 

o 

o 

o 


Convert each of the names into strings by surrounding the data with quotes. 

Separate each of the list items from the next with a comma. 

Surround the list of items with opening and closing square brackets. 

Assign the list to an identifier (movies in the preceding code) using the 
assignment operator (=). 


It’s perfectly OK to put your list creation code all on one line , assuming, of 
course, that you have room: 



This works, ioo. 


you are here ► 


7 









not my type 



Hang on a second! Arerft you 
forgetting something? Don't you need to 
declare type information for your list? 


No, because Python’s variable identifiers 
dorTt have a type. 

Many other programming languages insist that every 
identifier used in code has type information declared for 
it. Not so with Python: identifiers are simply names that 
refer to a data object of some type. 

Think of Python 5 s list as a high-level collection. The 

type of the data items is not important to the list. It’s 
OK to state that your movi es list is a “collection of 
strings,” but Python doesnP need to be told this. All 
Python needs to know is that you need a list, you Ve 
given it a name, and the list has some data items in it. 
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meet python 


Lists are like arrays 

When you create a list in Python, the interpreter creates an array-like data 
structure in memory to hold your data, with your data items stacked from 
the bottom up. Like array technology in other programming languages, the 
first slot in the stack is numbered 0, the second is numbered 1, the third is 
numbered 2, and so on: 


TWis is '/ouv 
'Vovies” V«st m 



TV»is is youV" wovics 
list i* 


data i-tcrh 
i» the list has a 

nurnevic OFFSBT 
assotisied With it. 


Access list data using the square bracket wotatiow 

As with arrays, you can access the data item in a list slot using the Standard 
square bracket offset notation : 


s-tavts douirrtm^ 

-Cvom z£V-o- 



da-ta i*tcw> oy \ sfcv-ee*. ^^ucs-tcd da-fca appears ok s^ree*. 


Let’s use IDLE to learn a bit about how lists work. 


you are here ► 
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idle session 


r\#7 An IDLE Session- 

Lists in Python might look like arrays, but they are much more than that: they are full-blown Python collectiori 
objects. This means that lists come with ready-to-use functionality in the form of list methods. 

Let's get to know some of Python's list methods. Open up IDLE and follow along with the code entered at the »> 
prompt. You should see exactly the same output as shown here. 

Start by defining a list of names, which you then display on screen using the print () BIF.Then, use the len () 
BIF to work out how many data items are in the list, before accessing and displaying the value of the second data 
item: 


»> cast = ["Cleese", 'Palin', ' Jones' , "Idle"] 

»> print (cast) 

['Cleese', 'Palin', 'Jones', 'Idle'] 

|t’s OK to m«lce a BlU 
the vesults o-f anothev o\>- 

Palin 


»> print (len (cast)) 


»> print (cast [1]) 


With your list created, you can use list methods to add a single data item to the end of your list (using the 
append () method), remove data from the end of your list (with the pop () method), and add a collection of 
data items to the end of your list (thanks to the extend () method): 


! 


»> cast.append("Gilliam") 
»> print (cast) 

['Cleese', 'Palin', 'Jones' 

»> cast.popO 

Gilliam' 

»> print (cast) 

['Cleese', 'Palin', 'Jones' 

»> cast. extend ([ "Gilliam" , 
»> print (cast) 

['Cleese', 'Palin', 'Jones' 


/Wethods are invoked MS i h g -the 
£ommon dot hotatioh. 

'Idle', 'Gilliam'] 



li-s aether list iW se\>avated ty 
suv-rowvded loy s<\wave Watkets- 


'Idle', 'Gilliam', 'Chapman'] 


Finally, find and remove a specific data item from your list (with the remove () method) and then add a data item 
before a specific slot location (using the insert () method): 


»> cast. remove ("Chapman") 

»> print (cast) 

['Cleese', 'Palin', 'Jones', 'Idle', 'Gilliam'] 
»> cast. insert (0 , "Chapman") 

»> print (cast) 

['Chapman', 'Cleese', 'Palin', 'Jones', 'Idle', 


A-Pbr ali that, we e«d up with 
xhe Cdsi o-f /VJoh-fcy Py-fch< 

Flyihg Cimius/ 


ioh s 


'Gilliam'] 
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meet python 


Add more data to your list 

With your list of movie names created, now you need to add more of the 
movie buff 5 s complex data to it. You have a choice here: 




Either strategy works. Which works best for you depends on what you are 
trying to do. Let 5 s recall what the movie buff’s data looks like: 


/\ numbev- 

veyv-esent»^ 

-the yeav 



The Lite of Bv-i,| 



The Holy ^yaily fpO ) rv-y Jones f Tev-vy ^illia», *}l mins 
£jv-aham Chapman 

/VJidhael Palm, John Cleese, Teyyy £jilliam, Tvid Idle f Teyv-y Jones 
vy Jones, mins 
^raham Uhafman 

/VTidhael Palm, John Cleese, Te*-v-y ^illiam, £*-id Idle f Tevv-y Jones 
The Aleanin^ of Lif( l^03>,h ev-vy Jones, 107 mins 
The si* Mon-ty Python das-1 membev-s 

<^yaham Chapman, /Vlidhael Palin, John Cleese, Tevv-y ^jilliam, Tv-id Idle f 7evv-y Jones 



The next piece of data you need to add to your list is a number (which 
represents the year the movie was released), and it must be inserted after each 
movie name. Let 5 s do that and see what happens. 


you are here ► 
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mixed type 



No, not madness, just the way Python works. 

Python lists can contain data of mixed type. It’s perfectly OK 
to mix strings with numbers within the same Python list. In fact, 
you can mix more than just strings and numbers; you can store 
data of any type in a single list, if you like. 

Recall that a Python list is a high-level collection, designed from 
the get-go to store a collection of “related things.” What type 
those things have is of little interest to the list, because the list 
exists merely to provide the mechanism to store data in list form. 

So, if you really need to store data of mixed type in a list, 

Python won’t stop you. 
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meet python 



Let’s take a bit of time to try to work out which strategy to use when adding data to your list in 
this case. 

Given the following list-creation code: 




Work out the Python code required to insert the numeric year data into the preceding list, 
changing the list so that it ends up looking like this: 


["The Holy Grail", 1975, "The Life of Brian", 1979, "The Meaning of Life", 1983] 


\AM it 70UV 

mse\rt»or\ - ^ 

todt here- 



Now write the Python code required to re-create the list with the data you need ali in one go: 


Mriie your 

're-£\rea'fciofi ^ 

todt here. 


In this case, which of these two methods do you think is best? (Circle your choice). 



you are here ► 
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additional data 



ExeftciSe 


Solitiori 


Let’s take a bit of time to try and work out which strategy to use when adding data to your list in 
this case. 

Given the following list-creation code: 


movies = ["The Holy Grail", "The Life of Brian ", "The Meaning of Life"] 



You were to work out the Python code required to insert the numeric year data into the preceding 
list: 


Insert the f rst year 

WcOVt setottd list —. 

•,te**. ^ movies.'mse\rtO, 1^75) 


/hscv**t the scc.oY\d yca^ 

BE fORB the -Pouv-th list item.-> movics.insertft> \°[}°0 ^ 


Pid you get the »,ath v-ight? 
Micr -the fhrst insection, the 
list grows, so you have to take 
that mto Consideratio* \wheh 
working out where to do the 
seeohd ihscvt- 


movics.apper\dO 


The* appe*d the last yeav- 
.the CY\d of the 'list' - 



^ You were also to write the Python code required to recreate the list with the data you need ali in 
one go: 


movies =■ C”The Holy £jvail”, l°n^i 

w . )> w The Li-fe of Bria*”, \° 0 °\, 

Assia* all youv data to the "<>v,es .. 

idetvti^ier. What *as prev.ously there * "The 0 £ Li&" 1^033 

veplated- 

In this case, which of these two methods do you think is best? (You were to circle your choice.) 

Yes, met^od 2- ****? 

oytioh here...tV>at is, +or a s»all 

list like this- Also, there s no 

tv-ieky douifttm^ to do. 
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meet python 


Work with your list data 

You often need to iterate over your list and perform some action on each item 



This code works as expected, making the data from the list appear on screen. 
However, if the code is later amended to add another favorite movie to the list, 
the list-processing code stops working as expected, because the list-processing code 
does not mention the third item. 

Big deal: all you need to do is add another print () statement, right? 

Yes, adding one extra print () statement works for one extra movie, but 
what if you need to add another hundred favorite movies? The scale of the 
problem defeats you, because adding all those extra print () statements 
becomes such a chore that you would rather find an excuse not to have to do. 


If s time to Iterate 


Processing every list item is such a common requirement that Python makes it 
especially convenient, with the built-in for loop. Gonsider this code, which is 
a rewrite of the previous code to use a for loop: 


Use 'V t» 
over Vtft, 


Pf"* a ^ poPula-fc e ;i 

M * you did b J 0 ,e * 


tbe vaWe <£ eatV^ 

mdWi(M «t®* on 

s cxttr\ as 70U Ofl' 



fav_movies = ["The Holy Grail", "The Life of Brian"] 



for each_flick in favjnovies: 
print(each flick) 



This is the list-yv-ottssi^ 
dode, usm$ a -for looy. 


Using a for loop scales and works with any size list. 


you are here ► 
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list Processing 


For loops work with lists of awy si ze 


Python 5 s for loop exists to process lists and other iterations in Python. Lists are 
the most common iterated data structure in Python, and when you need to 
iterate a list, it’s best to use for: 

The keywov-d *f° v ‘ 

'mditaies the start 

ot the loop and 
tomes loetove the 
-tav-oet identitiev- 

^for 


Tke teb-r™“T»“ de 

MWST v>e mdented 
wder the W W' 




The keyword w m sepav-ates 
the tardet idehti-Piev* -from 
your list- v 

in 



A £oloh -follows youv- 
list hame ahd ihdieates 
the start ot youv* list— 
pv-oeessihg eode- 



The list-processing code is referred to by Python programmers as the suite. 

The target identifier is like any other name in your code. As your list is 
iterated over, the target identifier is assigned each of the data values in your 
list, in turn. This means that each time the loop code exeeutes, the target 
identifier refers to a different data value. The loop keeps iterating until it 
exhausts all of your list’s data, no matter how big or small your list is. 

An alternative to using for is to code the iteration with a while loop. 
Gonsider these two snippets of Python code, which perform the same action : 


VVhch you use u while , 
you have to worry about 
U state iwCormatioh, 
whieh requires you 
to employ a eouhtmj 
idehtitiev- V. 



count = 0 

while count < len(movies): 
print(movies[count]) 
count = count+1 



These while and for statements do the same thing. 


for each_item in movies: 
print(each item) 


Wheh you use w for" the 
Pythoh interpreter 
worries about the “state 
ih-rormatioh^ -for you. 
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meet python 


O: So...when iterating over a list, I 
should always use for instead of while? 


Yes, unless you have a really good 
reason to use (or need the extra control 
of) a while loop. The for loop takes care 
of working from the start of your list and 
continuing to the end. It’s next to impossible 
to get stung by an off-by-one error when you 
use for. This is not the case with while. 


O: So, lists arerYt really like arrays 
then, because they do so much more? 


WelL.they are in that you can access 
individual data items in your list with the 
Standard square bracket notation, but—as 
youVe seen—Python’s lists can do so much 
more. At Head First Labs, we like to think of 
lists as “arrays on steroids.” 


Q: And they work this way only in 
Python 3, right? 


No. There are certain enhancements 
to lists that were added in Python 3, but 
release 2 of Python has lists, too. AII of what 
youVe learned about lists so far will work 
with lists in Releases 2 and 3 of Python. 


Why are we using Python 3? What’s 
wrong with Python 2, anyway? Lots of 
programmers seem to be using it. 

Lots of programmers are using Python 
2, but the future of Python development lies 
with Release 3. Of course, moving the entire 
Python community to Python 3 won’t happen 
overnight, so there’s an awful lot of projects 
that will continue to run on Release 2 for the 
foreseeable future. Despite 2’s dominance 
at the moment, at Head First Labs we think 
the new bits in 3 are well worth the added 
investment in learning about them now. 

Don’t worry: if you know 2, Python 3 is easy. 


tWeiare no 

Dumb Quesfions 


% Seeing as Python’s lists shrink and 
grow as needed, they must not support 
bounds-checking, right? 


Well, lists are dynamic, in that they 
shrink and grow, but they are not magic, 
in that they cannot access a data item 
that does not exist. If you try to access a 
nonexistent data item, Python responds with 
an IndexError, which means “out of 
bounds.” 


Q: What’s with all the strange 
references to Monty Python? 


Ah, you spotted that, eh? It turns 
out that the creator of Python, Guido van 
Rossum, was reading the Scripts of the 
Monty Python TV shows while designing his 
new programming language. When Guido 
needed a name for his new language, he 
chose “Python” as a bit of a joke (or so the 
legend goes). 


% Do I need to know Monty Python in 
order to understand the examples? 


No, but as they say in the official 
Python documentation: “it helps if you do.” 
But don’t worry: you’ll survive, even if you’ve 
never heard of Monty Python . 


% I notice that some of your strings 
are surrounded with double quotes and 
others with single quotes. What’s the 
difference? 


There isn’t any. Python lets you use 
either to create a string. The only rule is that 
if you start a string with one of the quotes, 
then you have to end it with the same 
quote; you can’t mix’n’match. As you may 
have seen, IDLE uses single quotes when 
displaying strings within the Shell. 


O: What if I need to embed a double 
quote in a string? 

You have two choices: either escape 
the double quote like this: \ ", or surround 
your string with single quotes. 

O: Can I use any characters to name 
my identifiers? 

No. Like most other programming 
languages, Python has some rules that 
must be adhered to when creating names. 
Names can start with a letter character or 
an underscore, then include any number 
of letter characters, numbers, and/or 
underscores in the rest of the name. Strange 
characters (such as %$£) are not allowed 
and you’11 obviously want to use names that 
have meaning within the context ofyour 
code. Names like members, the_ 
time , and people are much better 
than m, t, and p, aren’t they? 

O: Yes, good naming practice is 
always important. But what about case 
sensitivity? 

Yes, Python is the “sensitive type,” in 
that Python code is case sensitive. This 
means that msg and MSG are two different 
names, so be careful. Python (and IDLE) 
will help with the problems that can occur as 
a resuit of this. For instance, you can use 
an identifier in your code only if it has been 
given a value; unassigned identifiers cause 
a runtime error. This means that if you type 
mgs when you meant msg, you’ll find out 
pretty quickly when Python complains about 
your code having a Na me Error. 


you are here ► 
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lists within lists 


Storc lists within lists 


As youVe seen, lists can hold data of mixed type. But it gets even better than 
that: lists can hold collections of anything, including other lists. Simply embed 
the inner list within the enclosing list as needed. 

Looking closely at the movie buff’s data, it is possible to determine a structure 
which looks much like a list of lists: 


Thev-e s ohly ov\e \ead 
d£-fco\r lisied here, but 
thcirc £ould bc 


Ueres a \\si o-C r*ov\e 


...wbidb itscl-f dohtams 
a list o( Icad atbors.' 



The ttoly 6\ra\\, \°f\ 5, Tev-iry Jones f Tcv-vy £jilliam, *)l mins 
£jv-aham Cbapman 

Midbacl Palm, J oh* Cleese, Terry £jilliam, Evid Idle f Terry Jones 



• whi£h rfcsel-f 
^Ohtaihs a list o-p 
suppov-tihj atbors. 


In Python, you can turn this real list of data into code with little or no effort. 
All you need to remember is that every list is a collection of items separated 
from each other with commas and surrounded with square brackets. And, of 
course, any list item can itself be another list: 



The stavt of -the 
•fivsti outev list 


f- 


movies = [ 

"The Holy Grail", 1975, "Terry Jones & Terry Gilliam", 91, 
["Graham Chapman", 


The end of a!! the 
lists is heve- 



L 


["Michael Palin", "John Cleese", "Terry Gilliam" 


The stavt of the 
setotid, ihhClr |ist : 
“moviesC^pJ” 


The stavt of the thivd, m«ev 
'mnev list : tt moViesC^r3CI 3 



So, a list within a list is possible, as is a list within a list within a list (as this 
example code demonstrates). In fact, it’s possible to nest lists within lists to 
most any level with Python. And you can manipulate every list with its own list 
methods and access it with the square bracket notation: 


"Eric Idle", "Terry Jones"]]] 


This looks a little weivd...u*til you 
veme*«bev that theve ave thvee 

TT 9 fi"** bvaekets, so theve ™st 
also be thvee elosing ones. 


print(movies[ 4 ][ 1 ][ 3 ]) 

T' 

\ 


Eric Idle 


f\ list Viithm a list xi-thin 3 list 
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An IDLE Session 


Creating a list that contains another list is straightforward. But what happens when you try to process a list that 
contains another list (or lists) using the for loop from earlier in this chapter? 

Let's use IDLE to work out what happens. Begin by creating the list of the movie data for "The Holy Grail" in 
memory, display it on screen, and then process the list with your for loop: 

»> movies = ["The Holy Grail", 1975, "Terry Jones & Terry Gilliam", 91, 

["Graham Chapman", ["Michael Palin", "John Cleese", 

"Terry Gilliam", "Eric Idle", "Terry Jones"]]] 

»> print (movies) 

['The Holy Grail', 1975, 'Terry Jones & Terry Gilliam', 91, ['Graham Chapman', ['Michael Palin', 
'John Cleese', 'Terry Gilliam', 'Eric Idle', 'Terry Jones']]] 

»> for each item in movies: 3 list WrlWm 

print (each_item) ' a ^3S bee* £ve3rbed 

rv>C**ovy- 

The Holy Grail 

1975 The w *Por w loop pr-in-b e3£h iiem o-P 

Terry Jones & Terry Gilliam ihe ouiev- loop ONL-Y- 

91 

['Graham Chapman', ['Michael Palin', 'John Cleese', 'Terry Gilliam', 'Eric Idle', 'Terry Jones']] 


A 


The innev* lis-t wrthin *thc ivmev- lis*t is pvivrted 3 s-is 



Yes, that’s correct: the loop code isn’t complete. 

At the moment, the code within the loop simply prints each list 
item, and when it finds a list at a slot, it simply displays the entire 
list on screen. After ali, the inner list is just another list item as far as 
the outer enclosing list is concerned. What 5 s we need here is some 
mechanism to spot that an item in a list is in fact another list and take 
the appropriate action. 

That sounds a little tricky. But can Python help? 


you are here ► 
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looking for lists 


Check a list for a list 


Each time you process an item in your list, you need to check to see if the 
item is another list. If the item is a list, you need to process the nested list 
before processing the next item in your outer list. Deciding what to do when in 
Python follows the familiar if... else... pattern: 

The keywovd 

indidates the sta^t 
o-f the dedision dode 

S* 



A Lo\ov\ (0 -follows your 
^ohdrtioh test. 



Th>s dode e^tes th <> 

jd-.„ Uo ds u e > 


[ his < '® de exedutes i+ the dond 
does HOT hold (U, FALSI 



Kote : hoth suites 
are mdcwtcd- 


No surprises here, as the if statement in Python works pretty much as 
expected. But what condition do you need to check? You need a way to 
determine if the item currently being processed is a list. Luckily, Python ships 
with a BIF that can help here: isinstance () . 

What 5 s cool about the isinstance () BIF is that it lets you check if a 
specific identifier holds data of a specific type: 


Create a shor-fc list a*d 
assi 9 h it to a* identitiev. 




An (DLE Session 


Let's use the IDLE shell to learn a little about how isinstance () works: 


Ask i-f W haw»cs w is a list (it is). 

Assiy» a numbev to an 


»> names = [' Michael' , ' Terry' ] 

( »> isinstance (names, list) 

True «C_^ Retev to a Python ty?e 

^ »> ™ nam „ - l©n fnames^ /- he«- |n thiS dase, the 

TC* \ ~ ^ is w list • 

dentilrier. | »> isinstance (num names, list) 

h 


Ask .t V ' ,sa 
Vist (it t^- 


False 
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meet python 



Here’s a copy of the current list-processing code. Your task is to rewrite this code using an if 
statement and the isinstance () BIF to process a list that displays another list. 



VVvi-b t youv- 

y\£V/ CoAt -^ 


tJiereiare no 

Durnb Qiiesti°ns 


q Are there many of these BIFs in 
Python? 


A: 


Yes. At the last count, there were over 
70 BIFs in Python 3. 

O: Over 70! How am I to remember 
that many, let alone find out what they all 
are? 


A 


You don’t have to worry about 
remembering. Let Python do it for you. 


0 : 


How? 


At the Python or IDLE shell, type 

dir (_builtins _) to see a list 

of the built-in stuff that comes with Python 
(that’s two leading and trailing underscore 
characters, by the way). The shell spits 
out a big list. Try it. AII those lowercase 
words are BIFs. To find out what any BIF 
does—like input (), for example—type 
help(input) at the shell for a 
description of the BIFs function. 



Why so many BIFs? 


Why not? Because Python comes with 
lots of built-in functionality, it can mean less 
code for you to write. This Python philosophy 
is known as “batteries included”: there’s 
enough included with Python to let you do 
most things well, without having to rely on 
code from third parties to get going. As well 
as lots of BIFs, you’ll find that Python’s 
Standard library is rich and packed with 
features waiting to be exploited by you. 
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list the list 



ExwctSe 


§0(.|#tl0H 


Here’s a copy of the current list-processing code. Your task was to rewrite this code using an if 
statement and the isinstance () BIF to process a list that displays another list. 


for each_item in movies 
print(each item) 


Pv-otess -the “movies 
list as be-fove- 



-Por eadb item in movies: 


You need to dbeC-k it tbe w . f* . . •/ i* m ,Mcr 

1 , , . -^ it ismstandeCeadh item, list/ : # 

duvv-ent item is a list- ... .... y 

It it is a list, use another._w .5 

llt” ^ ^ Vl pmt(nestedjW 

else: 


The ihhev- loop 

^eeds 3 hew tav-aet 
ideivfci-fier. 


Pid 


you «.anage to 9 et you^ p,int(eadh item), 
mdehtation vight? . . T. . 


|.f the dwvvent item 
o-f the entlosin^ list 
isnt a list, display it 
on screem 


p IT* / An IDLE Session 



Let's use IDLE to see if this code makes a difference to the output displayed on screen: 

»> for each_item in movies: 

if isinstance(each_item, list): 
for nested_item in each_item: 
print(nested_item) 

else: 

print(each_item) 

The Holy Grail 
1975 

Terry Jones & Terry Gilliam 
91 

Graham Chapman 

['Michael Palin ', 'John Cleese ', 



This is a little tettev, but not by 
n>wth...theve , s anothev nested list here 
that's not beinO) protessed prope,!?. 


I 


'Terry Gilliam', 'Eric Idle', 'Terry Jones'] 
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meet python 


Complex data is hard to process 

The movie buff’s data is complex. Let 5 s take another look at a subset of the 
data and your Python code that processes it. 

The ou-tev-, endlosm^ lis*t 









Can you spot the problem with your Python 
code as it is currently written? What do you 
think needs to happen to your code to allow it to 
process the movie buffs data correctly? 


Yeah.. that's almost 
working. ..it's just a 
pi+y about that list of 
supporting actors... 



you are here ► 
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nested lists 


Handle wany levels of nested lists 

The data and your code are not in sync. 

The movie buff’s data is a list that contains a nested list that itself contains 
a nested list. The trouble is that your code knows only how to process a list 
nested inside an enclosing list. 

The solution, of course, is to add more code to handle the additionally nested list. By 
looking at the existing code, it’s easy to spot the code you need to repeat: 
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meet python 




I just love that...in fact,1 love it so 
much I've decided to add another list to my 
data. I want to include the other movies each 
supporting actor has starred in. If I add the 
data, can you change your code to print this 
data, too? 


That’s more list data and more Python code. 

The data has to be embedded as another nested list within the already deeply 
nested list of supporting actors. That 5 s possible to do, even though it makes 
your head hurt just to think about a list of lists of lists of lists! Amending your 
code is just a matter of adding another for loop and an if statement. 

That doesiVt sound like too much trouble, does it? 

you are here ► 25 











avoid complexity 



Adding another nested loop is a huge pain. 

Your data is getting more complex (that mind-bending list 
of lists of lists of lists) and, as a consequence, your code is 
getting overly complex, too (that brain-exploding for loop 
inside a for loop inside a for loop). And overly complex 
code is rarely a good thing... 
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meet python 



Wouldnt it be dreamy if there were an 
efficient way to process lists, preferably 
using a technique that resulted in less code x 
not more? But I know it's iust a fantasy... 
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reduce, reuse, recycle 


P(m't rcpcat code; create a function 


Take a look at the code that youVe created so far, which (in an effort to save 
you from havin gyour brain explode) has already been amended to process yet 
another nested list. Notice anything? 



Your code now contains a lot of repeated code. It 5 s also a mess to look at, even 
though it works with the movie buff 5 s amended data. All that nesting of for 
loops is hard to read, and it 5 s even harder to ensure that the else suites are 
associated with the correct if statement. 

There has to be a better way.. .but what to do? 

When code repeats in this way, most programmers look for a way to take 
the general pattern of the code and turn it into a reusable function. And 
Python programmers think this way, too. Greating a reusable function lets 
you invoke the function as needed, as opposed to cutting and pasting existing 
code. 

So, let’s turn the repeating code into a function. 
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meet python 


Create a fimctiow m Python 


A function in Python is a named suite of code , which can also take an optional 
list of arguments if re quire d. 


You define a Python function using the def statement, providing a name 
for your function and specifying either an empty or populated argument list 
within parentheses. The Standard form looks something like this: 


The keyw>v-d w def 
m-tv-odufces the *aw>e 
of -the ‘funfction- 




/\v^ume^*t lis-ts are optional, 
but the parentheses are NOT. 



TV>e WW s toAt r 
MlAST V>e 'mdented <*« 
det state»»*. 



A doloh 6) -follows -the 
^losihj parenthesis 
a*d ihdiea-tes -the 
of your 

-fund-tiohs dode sui-te. 



What does your function need to do? 

Your function needs to take a list and process each item in the list. If it finds a 
nested list within the first list, the function needs to repeat. It can do this by 
invoking itself on the nested list. In other words, the function needs to recur— 
that is, invoke itself from within the funtion code suite. 



Let's call the function that you'll create print_lol (). It takes 
one argument: a list to display on screen. Grab your pencil and 
complete the code below to provide the required functionality: 


def print lol (the list) : 


for 

if 


else: 
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recursive function 


r /T,r: 

Solution 


You were to call the function that you'll create print_lol (). It 
takes one argument: a list to display on screen. You were to grab your 
pencil and complete the code to provide the required functionality: 


def print_lol(the_list): 

Pv-otess the ' for eac |, ite™ i. the liet is itself s |T"'' TiT** 

list «*h 3 l~f. - .-. IT* ' W ' " ,<4 ' 

if isinstanCe(eath_ite»n, list)-' 

print_lol(each_ite»n) 

else: _ |£ the item bein^ proeessed ISKT 

print(eadh_item) y L aisvlav the item on sereen- 



? j An IDLE Session- 

Let's use IDLE one final time to test your new function. Will it work as well as your earlier code? 

»> def print_lol (the_list) : 

for each_item in the_list: 

if isinstance(each_item, list) 
print_lol(each_item) 

else: 

print(each_item) 

/«voke the -function. 


Ii Works, too/ The redusrive -function 

produces EXACTL/ the same results as 
CJvliev £ode. 


»> print_lol (movies) 

The Holy Grail 
1975 

Terry Jones & Terry Gilliam 
91 

Graham Chapman 
Michael Palin 
John Cleese 
Terry Gilliam 
Eric Idle 
Terry Jones 
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meet python 


Recursion to the rescue! 

The use of a recursive function has allowed you to reduce 14 lines of messy, 
hard-to-understand, brain-hurting code into a six-line function. Unlike the 
eadier code that needs to be amended to support additional nested lists 
(should the movie buff require them), the recursive function does not need to 
change to process any depth of nested lists properly. 

Python 3 defaults its recursion limit to 1,000, which is a lot of lists of lists of lists 
of lists.. .and this limit can be changed should you ever need even more depth 
than that. 


What a great start! 



Ah, y es, that's terrif ic! 
re\ax, knowing that your 
process my movie data, 
shouldve done this vea 


I 


can 


now 


code 


can 


really 


I 


years 


ago 


By taking advantage of functions and recursion, you Ve solved the code 
complexity problems that had crept into your earlier list-processing code. 

By creating print_lol (), you Ve produced a reusable chunk of code that 
can be put to use in many places in your (and others) programs. 


You’re well on your way to putting Python to work! 
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CHAPTER 1 


python toolbox 



Your Pythow Toolbox 

You’ve got Chapter 1 under your 
belt and you’ve added some key 


Python goodies to your toolbox. 



BULLET POINTS 


Run Python 3 from the command line or 
from within IDLE. 


Identifiers are names that refer to data 
objects. The identifiers have no “type,” but 
the data objects that they refer to do. 


PytW Umjo 

9 "BIF - a Wtl-t-m WW 

9 "Suite" - a Wotk <£ Pytko» tode ’ 
is mdewted to mditate yco»^- 

9 "Battev-ies intluded - a ^ay 

to tke £aet tkat PytW to^s 

viitV» most everytVm^ y^'" " ee<i ^ 5 e 
ojomO) <v*itkly a*d pv-oduttWely- 


IDLB 


@ The IDLE Shell lets you expev-ir^rT *i{ 
youir tode as you wi-fce it 

o Adjust IDLEs pireteire^ees to suit the 

y° u 

o wher, v/orkir^ with the shell, 

use Alt-p 4r Pirevious ar>d use Alt-tf 4. 
^ext fbut use Ctrl i-P y ou Ve or. a Mat). 


print() BIF displays a message on 
screen. 

A list is a collection of data, separated 
by commas and surrounded by square 
brackets. 

Lists are like arrays on steroids. 

Lists can be used with BIFs, but also 
support a bunch of list methods. 

Lists can hold any data, and the data can be 
of mixed type. Lists can also hold other lists. 

Lists shrink and grow as needed. AII of the 
memory used by your data is managed by 
Python for you. 

Python uses indentation to group statements 
together. 

len () BIF provides a length of some data 
object or count the number of items in a 
collection, such as a list. 

The for loop lets you iterate a list and 
is often more convenient to use that an 
equivalent while loop. 

The if... else... statement lets you make 
decisions in your code. 

isinstanceO BIF checks whether 
an identifier refers to a data object of some 
specified type. 

Use def to define a custom function. 
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Reusable code is great, but a shareable module is better. 

By sharing your code as a Python module, you open up your code to the entire Python 
community...and it’s always good to share, isn’t it? In this chapter, you’ll leam how to 
create, install, and distribute your own shareable modules. You’ll then load your module 
onto Python’s Software sharing site on the Web, so that everyone can benefit from your 
work. Along the way, you’ll pick up a few new tricks relating to Python’s functions, too. 


this is a new chapter 
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Iet’s share 


Ifs too good wot to share 

YouVe been showing your function to other programmers, and they like what 
they see. 



Yes, a function this good should be shared with the world. 

Python provides a set of technologies that make this easy for you, which 
includes modules and the distributiori Utilities : 

O Modules let you organize your code for optimal sharing. 

O The distributiori Utilities let you share your modules with the world. 


Let 5 s turn your function into a module, then use the distribution Utilities to 
share your module with the wider Python programming community. 
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sharing your code 


Turw your fuwctiow iwto a module 

A module is simply a text file that contains Python code. The main 
requirement is that the name of the file needs to end in . py: the Python 
extension. To turn your function into a module, save your code into an 
appropriately named file: 




tJiere iare no 

Dumb Qiiestfcns 


£al| 

■fchis -fi| c 

'Ws-fcem.py” 




What’s the best Python editor? 


The answer to that question really depends on who you ask. However, you 
can, of course, use any text editor to create and save your function’s code in a 
text file. Something as simple as NotePad on Windows works fine for this, as does 
a full-featured editor such as TextMate on Mac OS X. And there’s also full-fledged 
IDEs such as Eclipse on Linux, as well as the classic vi and emaes editors. And, 
as you already know, Python comes with IDLE, which also includes a built-in 
code editor. It might not be as capable as those other “real” editors, but IDLE is 
installed with Python and is essentially guaranteed to be available. For lots of 
jobs, IDLE’s edit window is all the editor you’ll ever need when working with your 
Python code. Of course, there are other IDEs for Python, too. Check out WingIDE 
for one that specifically targets Python developers. 




r 





Go ahead and create a text 
file called ne ster . py that 
contains your function code 
from the end of Ghapter 1. 
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modules repository 


Modules are everywhere 

As might be expected, yoir 11 find Python modules in lots of places. 


I'm preloaded with 
lots of modules in the 
Python Standard Library... 
and they are already on your 
computer. 



/ mu m\ 0 


If the Standard Library doesn't do 
it for you, why not try the Web? 

I hear PyPI is where third-party 
Python modules hang out. 


O 


P^Pl » s y*-oY\ovAY^ed 

The Python Package Index (or PyPI for short) provides a 
centralized repository for third-party Python modules on the 
Internet. When you are ready, you’11 use PyPI to publish your 
module and make your code available for use by others. And your 
module is ready, but for one important addition. 

What do you think is missing from your module? 




If you are already familiar with 
Perfs CPAN repository, you can 
think of PyPI as the Python 
equivalent. 
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sharing your code 


Cowwent your code 


It 5 s always a good idea to include comments with your code. As your plan to 
share your module with the world, well-written comments help to document 
your work. 

In Python, a common commenting technique is to use a triple quote for 




Pu i 


you(r module 


hev- e . ~\^ 


Here is your module code (which is saved in the file nester . py). In 
the spaces provided, use your pencil to compose two comments: the 
first to describe the module and the second to describe the function. 


def print lol (the list) : 


Addi a torr.»wt 
your WW 

\\cct- 


}. 

for each_item in the_list: 

if isinstance(each_item, list) 
print_lol(each_item) 

else: 

print(each item) 
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request for comments 


^r /T.r; 

Solution 


Here is your module code (which is saved in the file nester . py). In 
the spaces provided, you were asked to use your pencil to compose 
two comments: the first to describe the module and the second to 
describe the function. 



www 7Vis is w r\cs*tcv- py w module, a*d i*t provides or\e -Pund-tio* dalled 
prirrt_JolO whidh priwts lis-ts *tha*t may o\r may r\o*t 'mdlude r\es*ted lisis. www 

pid y ou def print_lol(the_list) : 

VCmCmbcr \P 

wtludc c tttttt This 'Pufidtiofi takes a posi-tio^al ar^umewb dalled w *tKc — list w , wW\cM is a*y 

Python list (o-P, possibly, ^es-ted lists). Eadh data item in the provided list 
is (redursively) printed to the sdreen on its ovm line. www 


for each_item in the_list: 

if isinstance(each_item, list) 
print_lol(each_item) 

else: 

print(each item) 



There are no ehanjes^to the 
aetual dode here; youVe >st 
addin^ some domments. 


q: How do I know where the Python 
modules are on my computer? 


Ask IDLE. Type import sys; 
sys . path (all on one line) into the IDLE 
prompt to see the list of locations that your 
Python interpreter searches for modules. 


O: Hang on a second. I can use to 
put more than one line of code on the 
same line in my Python programs? 


Yes, you can. However, I don’t 
recommend that you do so. Better to give 
each Python statement its own line; it makes 
your code much easier for you (and others) 
to read. 


tliereiare no 

Dumb Questipns 


Qj Does it matter where I put my 
nester. py module? 


For now, no. Just be sure to put it 
somewhere where you can find it later. In 
a while, you’11 install your module into your 
local copy of Python, so that the interpreter 
can find it without you having to remember 
when you actually put it. 


O: So comments are like a funny- 
looking string surrounded by quotes? 


Yes. When a triple-quoted string is 
not assigned to a variable, it’s treated like a 
comment. The comments in your code are 
surrounded by three double quotes, but you 
could have used single quotes, too. 


O: Is there any other way to add a 
comment to Python code? 


Yes. If you put a “#” Symbol anywhere 
on a line, everything from that point to the 
end of the current line is a comment (unless 
the “#” appears within a triple quote, in 
which case it’s part of that comment). A lot 
of Python programmers use the “#” Symbol 
to quickly switch on and off a single line of 
code when testing new functionality. 
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sharing your code 


r £ / An (DLE Session- 

U—/■ 

Now that youVe added your comments and created a module, let's test that your code is stili working properly. 
Rather than typing your function's code into IDLE's prompt, bring the nester . py file into IDLE's edit window, 
and then press F5 to run the module's code: 


Islotc “tV 

to\or LoAtA 


^ nester.py -/Users/barryp/HeadFirstPython/chapter2/nester.py 

"""This is the "nester.py" module and it provides one function called print_lol() 
which prints lists that may or may not include nested lists.""" 

def print lol (thelist): 

"""This function takes one positional argument called "the_list", which 

-- is any Python list (of - possibly - nested lists). Each data item in the 

^ provided list is (recursively) printed to the screen on it's own line.""" 

for each_item in the_list: 

if isinstance(each_item, list): 
print lol(each_item) 

else : 

print (each item) 

1 



Ln: 15 Coi: 0 



Nothing appears to happen, other than the Python shell "restarting" and an empty prompt appearing: 

»> ================================ RESTART ================================ 

»> 

»> 

What's happened is that the Python interpreter has reset and the code in your module has executed.The code 
defines the function but, other than that, does little else. The interpreter is patiently waiting for you to do 
something with your newly defined function, so let's create a list of lists and invoke the function on it: 

»> movies = [ 

"The Holy Grail", 1975, "Terry Jones & Terry Gilliam", 91, 

["Graham Chapman", 

["Michael Palin", "John Cleese", "Terry Gilliam", 



»> print_lol (movies) 
The Holy Grail 



/nvoke -the function on the list 


1975 

Terry Jones & Terry Gilliam 
91 

Graham Chapman 
Michael Palin 
John Cleese 
Terry Gilliam 
Eric Idle 
Terry Jones 


Cool- Y°uv Code ConWes to 
function as e*pected The data 
in the list of lists is displayed 
oy \ sdvccn. 
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distributiori plan 


Prepare your distribution 


In order to share your newly created module, you need to prepare a 
distribution. This is the Python name given to the collection of files that 
together allow you to build, package, and distribute your module. 

Once a distribution exists, you can install your module into your local copy 
of Python, as well as upload your module to PyPI to share with the world. 
Follow along with the process described on these two pages to create a 
distribution for your module. 

With the folder created, copy your ne ster . py module file into the 
folder. To keep things simple, let 5 s call the folder ne ster: 


Ronin h\/ rron+inn 


The 'Vestev-T/" 
module -file- " 





r 


D® thh! 



Follow along with each of 
the steps described on these 
pages. By the time you reach 
the end, your module will 
have transformed into a 
Python distribution. 



JThe hewly trcaicd 

Vster" (o\de<r U 
divedov-y). 



Create a file called “setup.py" in your new folder. 

This file contains metadata about your distribution. Edit this file by adding the following code: 



These ave -the 
values Head Fivs-t 
Labs use wi-th 
■theiv rmodules; 

youv rme-fcada-fca 

will be di-Pfeveht 
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sharing your code 


Puild your distributiori 


You now have a folder with two files in it: your module’s code in ne ster . py 
and metadata about your module in setup . py. Now, it 5 s time to build your 
distribution. 


^ Build a distribution file. 

The distribution Utilities include all of the smarts required to build a distribution. Open a terminal 
window within your nester folder and type a single command: python3 setup . py sdist. 


'wth YApyfk 2 i\' ^ 

'rythoh3/\p ythoK,. exe ". 


Eivtev- -the 
Lovnmand a-t 
the plrorupt. 


status mcssays 
ayyeavs ov\ sevecn» 

the 

creatio* 7<>uv 
distributio*- 


I File Edit Window Help Build 


$ python3 setup.py sdist 
running sdist 
running check 

warning: sdist: manifest template 'MANIFEST.in' does not exist 

warning: sdist: Standard file not found: should have README 

writing manifest file 'MANIFEST' 

creating nester-1.0.0 

making hard links in nester-1.0.0... 

hard linking nester.py -> nester-1.0.0 

hard linking setup.py -> nester-1.0.0 

creating dist 

Creating tar archive 

removing 'nester-1.0.0' (and everything under it) 

$ 



o 


Install your distribution into your local copy of Python. 

Staying in the terminal, type this command: sudo python3 setup. py install. 

| File Edit Window Help Install 


] 


A*other bu*£h ot 
status rmcssagcs 
appeam o* seree*, 
^ohtiv-i^ihg the 
i^stallatioh o-p 
youv distributio*. 


$ python3 setup.py install 

running install 

running build 

running build_j?y 

creating build 

creating build/lib 

copying nester.py -> build/lib 

running install_lib 

copying build/lib/nester.py -> /Library/Frameworks/Python. 
framework/Versions/3.l/lib/python3.1/site-packages 
byte-compiling /Library/Frameworks/Python.framework/Versions/3.1/ 
lib/python3.1/site-packages/nester.py to nester.pyc 
running install_egg_info 

Writing /Library/Frameworks/Python.framework/Versions/3.1/lib/ 
python3.1/site-packages/nester-l.0.0-py3.1.egg-info 


Your distribution is ready. 
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ready for distributiori 


A quick review of your distributiori 

Thanks to Python’s distributiori Utilities, your module has been transformed into 
a distribution and installed into your local copy of Python. 

You started with a single function, which you entered into a file called 
nester.py, creating a module. You then created a folder called ne ster 
to house your module. The addition of a file called setup . py to your 
folder allowed you to build and install your distribution, which has resulted 
in a number of additional files and two new folders appearing within 
your ne ster folder. These files and folders are all created for you by the 
distribution Utilities. 


Before Setup 


After Setup 



tode is in 
“this -pile. 


nester.py 
setup.py 


J 


YouV" rv\C‘td(id‘td ,s 
m tWis -file- 


J 


Vouir todt is ih 
this -file. 



y< 3 UV ,s 

m -tWis -file. 


MANIFEST 


A list o-f -files ih 

youv disVibu-tioh is 
■fi ibis -file. 



tteve ave youv 
kcw -foldevs. 



Youv toAt 
■fi "this -file 



nester.py 



/ 


This is youv 
disinbu-tioh 


nester-l.0.0.tar.gz 


nester.py 
nester.pyc yf 
setup.py 



A u £orhpiled” vevsioh 
°f youv c.odc is ih 
this -file. 


42 Chapter 2 




















sharing your code 


Import a module to use it 

Now that your module is built, packaged as a distribution, and installed, let 5 s 
see what’s involved in using it. To use a module, simply import it into your 
programs or import it into the IDLE shell: 



Ko*te : you do* ^eed “to 
mtMe tV>e >7” e*W<o* 
NMhe» yowv- modwle- 




The import statement telis Python to include the nester.py module in 
your program. From that point on, you can use the module 5 s functions as if 
they were entered directly into your program, right? Well.. .that’s what you 
might expect. Let’s check out the validity of your assumption. 



Write a small program that imports your newly created module, defines a small list called “cast,” 
and then uses the function provided by your module to display the contents of the list on screen. 
Use the following list data (all strings): Palin, Cleese, Idle, Jones, Gilliam, and Chapman. 


Open your program in IDLE’s edit window, and then press F5 to execute your code. Describe 
what happens in the space below: 


you are here ► 
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idle error 



ExeftciSe 


§0(.|#tl0H 


You were to write a small program that imports your newly created module, defines a small list 
called “cast,” and then uses the function provided by your module to display the contents of the 
list on screen. You were to use the following list data (ali strings): Palin, Cleese, Idle, Jones, 
Gilliam, and Chapman. 



Open your program in IDLE’s edit window, and then press F5 to execute your code. Describe 
what happens in the space below: 


Bu-fc it didh i wovk/ — --^ IDLE givcs a* error, a*\d ihe yroyrdivn does y\o i vW 


— f /An IDLE Session- 

With your program in the IDLE edit window, pressing F5 (or choosing Run Module from the Run menu) does 
indeed cause problems: 

nnn t ryjn e 5 te r. py - /Use rs / b arryp / H eacf F i rstlPytih on/e h ap ter2 /1 ry n e 5 te r. py 

import nester 

cast = ['Palin 1 , 'Cleese' f r Idle 1 , 'Jones 1 , 'Gilliam', r Chapman'] 
print_lol(cast) 

Ln: 6 Coi: 0 ^ 


Your program does not appear to have exeeuted and an error message is reported: 


»> 


»> 


RESTART 


Traceback (most recent call last): 

File "/Users/barryp/HeadFirstPython/chapter2/try_nester.py", line 4, in <module> 
print_lol(cast) 

NameError: name 'print_lol' is not defineci\ 


»> 


a7JZJ r z a :: ?, LB ’ R «»*• 

w» "* lik< 
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sharing your code 


Python s modules implement namespaces 


All code in Python is associated with a namespace. 

Code in your main Python program (and within IDLE’s shell) is associated 

with a namespace called_main_. When you put your code into its own 

module, Python automatically creates a namespace with the same name as 
your module. So, the code in your module is associated with a namespace 
called ne ster. 


Tha-ts a double uhdevscov-e i h 

W ot the wovd 'Wih" and 
a+ter i-fc. 



I guess namespaces are like family names? 

If someone is looking for Chris, we need to know 
if it's Chris Murray or Chris Larkin, right? The 
family name helps to qualify what we mean, as do 
namespace names in Python. 


Yes, namespace names are like family names. 

When you want to refer to some function from a module 
namespace other than the one you are currently in, you need 
to qualify the invocation of the function with the module 5 s 
namespace name. 

So, instead of invoking the function as print_lol (cast) 
you need to qualify the name as nes ter . print_lol (cast) 
That way, the Python interpreter knows where to look. The 
format for namespace qualification is: the module 5 s name, followed 
by a period, and then the function name. 


The module »ame, 'Nhith ( 

idexiities the «amesyate- 



The -fuhfrtioh is the» invoke 
«oirmal, with "east" pvovide 
■fcke list to pro£ess. 


period separans -the module »ar»es?aee 
y\dw\e 'frow» the tuttCtiott na^e- 


as 

as 


you are here ► 


45 












ready for pypi 



An (DLE Session 


Let's test this. Staying at the IDLE shell, import your module, create the list, and then try to invoke the function 
without a qualifying name. You're expecting to see an error message: 


»> import nes ter 

»> cast = ['Palin', 'Cleese', 'Idle', ' Jones 
»> print_lol (cast) 

Traceback (most recent call last) : 

File M <pyshell#4> M , line 1, in <module> 
printlol(cast) 

NameError: name 'print_lol' is not definecT 


! ! 



Gilliam', 'Chapman'] 

h tvietkA, 7 ^ t*At 

HanS+w, betausc you d»d* t T al, ^Y 
-the name- 


When you qualify the name of the function with the namespace, things improve dramatically: 



wt Mst ^ a*played oh sdveeh 



When you use a plain import statement, such as import nester, 
the Python interpreter is instructed to allow you to access nestets 
functions using namespace qualification. However, it is possible to be 
morespecific. Ifyouusefrom nester import print_lol, the 
specified function (print_lol in this case) is odded to the current 
namespace, effectively removing the requirement for you to use 
namespace qualification. But you need to be careful. If you already have 
a function called print_lol defined in your current namespace, the 
specific import statement overwrites your function with the imported 
one, which might not be the behavior you want. 


Your module is now ready for upload to PyPI. 
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sharing your code 


Register with the PyPI website 


In order to upload your distribution to PyPI, you need to register with the 
PyPI website. This is a relatively straightforward process. 

Begin by surfing over to the PyPI website at http://pypi.python.org/ and 
requesting a PyPIID: 


Pvovide the Uset-name 
you'd like -fco use... 


£»tev yowv tWe» yassv/ord 
twide to» - tontivw>atioirv 
yuvyoses. 


Manual user registration 



use "^tpythoh," beeause 
that Useirna^e is alveady taken. 


Pvovide a validi e^sil addvess. 


Lle* r* attwa h w 



have 


PGP Key ID 
(optional): 
Usage 
Agreement: 


po^t Wt to dlidk the 
I ag\ree” dhedkbox be&ve 
dlidkmg oh the Rc<\hicr 
buttoh. 



(This identifies a PGP or GPG key) 

By registering to upload content to PyPI, I agree and affirmatively acknowledge the following: 

1. Content is restricted to Python packages and related information only. 

2. Any content uploaded to PyPI is provided on a non-confidential basis. 

3. The PSF is free to use or disseminate any content that I upload on an unrestricted bc 
the web site are granted an irrevocable, worldwide, royalty-free, nonexclusive license 
the content, including in digital form. 

4. I represent and warrant that I have complied with ali govemment regulations concern 
particular, if I am subject to United States law, I represent and warrant that I have obt 
content I upload. I further affirm that any content I provide is not intended for use by < 
States Export Administration Regulations. 

01 agree 

Register 



If all of your registration details are in order, a confirmation message is sent 
to the email address submitted on the registration form. The email message 
contains a link you can click to confirm your PyPI registration: 


richard@python.org to me 

To complete your registration of the user 11 hf python" with the python module 
index s please visit the following URL 


htto[//ov oi . python .ora/ ov oi ?: action - u s er& otk=E1 1 Kd wfix 7Y 61 vfv G k H u i Ai z U bz f axx V H 


show details 




You are now registered with PyPI. 


Clidk “the 6orvCivr»aW 

Youv PyPI rcystration. 


you are here ► 
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register and upload 


Upload your code to PyPI 


You 5 re ready to rock! The code in your function has been placed in a module, 
used to create a distribution and installed into your local copy of Python. To 
upload your distribution to PyPI, complete these two steps: command-line 
registration with PyPI and command-line uploading. 

It might seem strange to have to register with PyPI again , seeing as you just 
did this with their website. However, the command-line uploading tool needs 
to be made aware of your PyPI Username and Password, and that 5 s what this 
registration does. Don’t worry: you have to do this only once. 


Instruet setup to 
register y OU v- de-fcaiis. 


'ou 


Con-Piv-m that y< 
want to use your 
just-£\reated PyPI 
^redentials. 



Use youv PyPI 
setbfijs and save 
-fov' -future 


then* -rov 


use- 




I File Edit Window Help Register 


$ python3 setup.py register 
running register 
running check 

We need to know who you are, so please choose either: 

1. use your existing login, 

2. register as a new user, 

3. have the server generate a new password for you (and email it to you), or 

4. quit 

Your selection [default 1]: 

1 

Username: hfpython 
Password: 

Registering nester to http://pypi.python.org/pypi 
Server response (200): OK 

I can store your PyPI login so future submissions will be faster. 

(the login will be stored in /Users/barryp/.pypirc) 

Save your login (y/N)?y 




With your registration details entered and saved, you are now ready to upload 
your distribution to PyPI. Another command line does the trick: 


lustrat sctw? jjf 
(justriWW to 'Y 1 


Setup eon-fiv-ims 

that the upload—-^ 

is su££ess-fu|. YOUV- 
dist\ribut ion is now 
pa\rt o( PyP|. 


I File Edit Window Help Upload 


$ python3 setup.py sdist upload 
running sdist 
running check 

reading manifest file 'MANIFEST' 
creating nester-1.0.0 
making hard links in nester-1.0.0... 
hard linking nester.py -> nester-1.0.0 
hard linking setup.py -> nester-1.0.0 
Creating tar archive 

removing 'nester-1.0.0' (and everything under it) 
running upload 

Submitting dist/nester-1.0.0.tar.gz to http://pypi.python.org/pypi 
Server response (200): OK 
$ 


Mob: If you tiry -to upload a «.odule called 

-1 
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sharing your code 


Welcowe to the PyPI cowwimity 

Gongratulations! You are now a full-fledged, card-carrying member of the 
PyPI community. Your distributiori has joined the over 10,000 other uploads 
on PyPI. Feel free to surf on over to the PyPI website to conflrm the upload. 

Programmers from all over the globe are now able to download, unpack, and 
install your module into their local copy of Python, which is pretty cool when 

you think about it. , 

You ve now wvitten and 

Published y°uv £°de.-- 

Sit back, put your feet up, and wait for the plaudits to ^ 6oo l - s 

begin... 


Q: Which is best: plain imports or 
specific imports? 


Neither, really. Most programmers 
mix and match based on their own personal 
preference and taste (although there are 
plenty of programmers willing to argue that 
their preferred way is the “one true way”). 


Note that the from module import 
function form pollutes your current 
namespace: names already defined in your 
current namespace are overwritten by the 
imported names. 

And when I press F5 in IDLE’s edit 
window, it’s as if the modulet code is 
imported with an import statement, right? 

Yes, that is essentially what happens. 
The code in your edit window is compiled 
and executed by Python, and any names 
in the edit window are imported into the 
namespace being used by IDLE’s Shell. This 
is handy, because it makes it easy to test 
functionality with IDLE. But bear in mind that 
outside of IDLE, you stili need to import your 
module before you can use its functionality. 


there j are no 

Dtimb Questfcns 


NL- Is it really necessary for me to 
install my modules into my local copy of 
Python? Can’t I just put them in any old 
folder and import them from there? 

Yes, it is possible. Just bear in mind 
that Python looks for modules in a very 
specific list of places (recall the import 
sys; sys . path trick from earlier 
in this chapter). If you put your modules 
in a folder not listed in Python’s path list, 
chances are the interpreter won’t find 
them, resulting in ImportErrors. Using the 
distribution Utilities to build and install your 
module into your local copy of Python avoids 
these types of errors. 



I noticed the distribution utiliites 


created a file called nester. pyc. 
What’s up with that? 


Thafs a very good question. When 
the interpreter executes your module code 
for the first time, it reads in the code and 
translates it into an internal bytecode format 
which is ultimately executed. (This idea is 
very similar to the way the Java JVM works: 
your Java code is turned into a class file as 


a resuit of your Java technologies compiling 
your code.) The Python interpreter is smart 
enough to skip the translation phase the next 
time your module is used, because it can 
determine when youVe made changes to 
the original module code file. If your module 
code hasn’t changed, no translation occurs 
and the “compiled” code is executed. If your 
code has changed, the translation occurs 
(creating a new pyc file) as needed. The 
upshot of all this is that when Python sees a 
pyc file, it tries to use it because doing so 
makes everything go much faster. 


O: Cool. So I can just provide my 
users with the pyc file? 


No, don’t do that, because the use of 
the pyc file (if found) is primarily a runtime 
optimization performed by the interpreter. 


O: So, can I delete the pyc file if I don’t 
need it? 


Sure, if you really want to. Just be 
aware that you lose any potential runtime 
optimization. 


you are here ► 
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conflicting requests 


With success comes respowsibility 


Lots of programmers from many different locations are using your module. 
And some of these programmers are looking for more features. 


We really love your code, 
but is there any chance this 
thing could print the data 
to screen and indent each 
nested list whenever one is 
found? 


O 


Likes wbat you ve 
dore, but dould 
be bayyier. 




A*y ehanges to the way your 
TUhdtioh works are likely to 
3hhoy this guy. 


Requests for change are inevitable 

You need to keep your current users happy by maintaining the existing 
functionality, while at the same time providing enhanced functionality to 
those users that require it. This could be tricky. 


What are your options here? 
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sharing your code 


life'$ full of choiccs 

When it comes to deciding what to do here, there 5 s 
no shortage of suggestions. 


Thafs soooo easy. Simply create 
another function called "print_lol2", 
right? You could then import the 
f unction you want using the specif ic 
form of the import statement. It's not 
that hard, really... 


Yeah, that might just work. 

You could edit your module’s code and define a new function called 
print_lol2, then code up the function to perform the nested printing 
When you want to use the original function, use this specific form of the 
import statement: from nester import print_lol. When you want 
to use the new, improved version of the function, use this import statement: 
from nester import print_lol2. 

Which would work, but... 


O 




Right. A second function is wasteful. 

Not only are you introducing an almost identical function to your 
module, which might create a potential maintenance nightmare, but 
you’re also making things much more difficult for the users of your 
module, who must decide ahead of time which version of the function 
they need. Adding a second function makes your module’s applicatiori 
programming interjace (API) more complex than it needs to be. 

There has to be a better strategy, doesrPt there? 


you are here ► 
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add an argument 


Control behavior with an extra argument 

If you add an extra argment to your function, you can handle indentation 
within your current code without too much trouble. 



Take your function to the next level 

At the moment, your function has a single argument: the_list. If you add 
a second argument called level, you can use it to control indentation. A 
positive value for level indicates the number of tab-stops to include when 
displaying a line of data on screen. If level is 0, no indentation is used; if 
it 5 s 1, use a single tab-stop; if it’s 2, use two tab-stops; and so on. 

It’s ciear you are looking at some sort of looping mechanism here, right? You 
already know how to iterate over a variably sized list, but how do you iterate a 
fixed number of times in Python? 

Does Python provide any functionality that can help? 
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sharing your code 


Pefore your write wcw code, thiwk PIF 

When you come across a need that you think is generic , ask yourself if there’s 
a built-in function (BIF) that can help. After ali, iterating a flxed number of 
times is something you’11 need to do ali the time. 

And remember: Python 3 includes over 70 BIFs, so there’s a lot of 
functionality waiting to be discovered. 



Use your pencil to draw a Ime matchmg each BIF to the correct descnption. 
The first one is done for you. Once you have all your lines drawn, circle the 
BIF you think you need to use in the next version of your function. 


BIF 



int () 


id () 


next() 


Wktthe BIF does 

Greates a numbered list of paired-data, starting 
from 0. 

Returns the unique identification for a Python 
data object. 

A factory function that creates a new, empty list. 


Returns the next item from a iterable data 
structure such as a list. 

Returns an iterator that generates numbers in a 
specified range on demand and as needed. 

Gonverts a string or another number to an 
integer (if possible). 


you are here ► 
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who did what 





+ 


BiF 


list () 


You were to use your pencil to draw a line matching each BIF to the correct 
description. Once you had all your lines drawn, you were to circle the BIF 
you think you need to use in the next version of your function. 

W}ictttJie BIF does 


Greates a numbered list of paired-data, starting 
from 0. 


TW.s W 

\ooVs 

enumerate () 



next() 


^ Returns the unique identification for a Python 
7 data object. 


A factory function that creates a new, empty list. 


Returns the next item from a iterable data 
structure such as a list. 


Returns an iterator that generates numbers in a 
specified range on demand and as needed. 


Gonverts a string or another number to an 
integer (if possible). 


The rangeO PIF iterates a fixed number of times 


The range () BIF gives you the control you need to iterate a specific 
number of times and can be used to generate a list of numbers from zero up 
to-but-not-including some number. Here 5 s how to use it: 


'W" is -the tavo,e-t idwfcvfiw 
and is assi<y>ed eatb ©t tl»e 
numbevs c^ev-a-ted by 
m h***' 



for num in range(4) 
print(num) 




^Chcvatc humbers up—-to— 
but—ho-fc— ih^ludihg 


The number 0 , 1,2, a«d % W.1I 

appcav oy \ screer*. 
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sharing your code 


there-iscce no 

Dumb Questi9ns 


q: Don’t I need to import the BIFs in 
order to use them in my program? 


No. For ail intents and purposes, the 
BIFs are specifically imported into every 
Python program as well as IDLE. 


O: So the BIFs must belong to the 
_main_namespace, right? 


No. They are automatically imported 

into the _ main _namespace, but the 

BIFs have their very own namespace called 
(waitforit) builtins 


nl- I get how range() works, but surely 
I could just as easily use a while loop to 
do the same thing? 

Yes, you can, but it’s not as elegant 
as using range (). Seriously, though, 
the while equivalent not only requires 
you to write more code, but it also makes it 
your responsibility to worry about loop state, 
whereas range () worries about this for 
you. As a general rule, Python programmers 
look for ways to reduce the amount of code 
they need to write and worry about, which 
leads to better code robustness, fewer errors, 
and a good night’s sleep. 



So BIFs are actually good for me? 


BIFs exist to make your programming 
experience as straightforward as possible 
by providing a collection of functions that 
provide common Solutions to common 
problems. Since they are included with 
Python, you are pretty much assured that 
they have been tested to destruction and 
do “exactly what it says on the tin.” You can 
depend on the BIFs. Using them gives your 
program a leg up and makes you look good. 
So, yes, the BIFs are good for you! 



Now that you know a bit about the range () BIF, amend your function to use range () to 
indent any nested lists a specific number of tab-stops. 

Hint: To display a TAB character on screen using the print () BIF yet avoid taking a new-line 
(which is print () ’s default behavior), use this Python code: print ("\t", end=' '). 


n n n 


This is the "nester.py" module and it provides one function called print_lol() 
which prints lists that may or may not include nested lists.""" 



UUe ihe ^ ^ e*bra a^u^t 


def print_lol(the_list, ): 

"""This function takes a positional argument called "the_list", which 

is any Python list (of - possibly - nested lists). Each data item in the 
provided list is (recursively) printed to the screen on it's own line.""" 


for each_item in the_list: 

if isinstance(each_item, list): 

print_lol(each_item) 
else: 


print(each item) 


t_ fordei -to cdi-t 

“the CorumeKvt. 



Add tode V\eve to take the 
number o( tab-sfco?s- 
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using range 



ExeRciSe 

§0(.|#tl0H 


Now that you know a bit about the range () BIF, you were to amend your function to use 
range () to indent any nested lists a specific number of tab-stops. 

Hint: To display a TAB character on screen using the print () BIF yet avoid taking a new-line 
(which is print () ’s default behavior), use this Python code: print ("\t", end=' '). 


"This is the "nester.py" module and it provides one function called print_lol() 
which prints lists that may or may not include nested lists.""" 


def print_lol(the_list, level ): 

"""This function takes a positional argument called "the_list", which 

is any Python list (of - possibly - nested lists). Each data item in the 
provided list is (recursively) printed to the screen on it's own line.^^^ 

A se£or\d av<yAr*errt ealled u level” is used *bo 'msev*t *tab-S'boj>s v/her\ a r\es*ted list is er\£ourrteved. 

for each item in the list: 


)))w 


if isinstance(each_item, list): 
print lol (each item) 


else: 


•fov tab__stop 'm v-a^cOcvcl): 



Use -the value w l t*t\’ “to dontvol 
r»a*y tab-stop ave used- 


. e d - 1 . f M ibtaefa.for eaeh | eve | 

print (each item) oT ihdeirtati 


:iom. 


■ ” / An (DLE Session 



lt's time to test the new version of your function. Load your module file into IDLE, press F5 to import the function 
into IDLE's namespace, and then invoke the function on your movies list with a second argument: 

^— Invoke youv -PuKe-fcioh, being suve -fco pvovide 

»> print_lol (movies , 0) a sedohd avgur^eht. 

The Holy Grail \ 

1975 V._The data m "movies" stav-ts to appeav o* stveen... 

Terry Jones & Terry Gilliam 

91 s 

Traceback (most recent call last) : ^ ” — tbem c bveaks loose/ Sornethihg is hot Htjht heve- 

File M <pyshell#2 > M , line 1 , in <module> 
print_lol(movies , 0) 

File M /TJsers/barryp/HeadFirstPython/chapter2/nester/nester.py M , line 14, in print_lol 
-rn^nt lol (each_item) , , 

_ne ve s youv elue as -fco 

print lol() takes exactly 2 positional arguments (1 given) -. i> 

—-—- M i wnats gone v/vonjy 
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sharing your code 


Python fries its best to run your code 

Unlike compiled languages (such as G, Java, C#, and others), Python doesn’t 
completely check the validity of your code until it runs. This allows Python to do 
some rather cool things that just aren’t possible in those other languages, such 
as dynamically defining functions at runtime. This, of course, can be very 
flexible and powerful. 

The cost to you, however, is that you need to be very careful when writing 
your code, because something that typically would be caught and flagged as 
an “error” by a traditional, statically typed, compiled language often goes 
unnoticed in Python. 


0 


...OK, C++ syntax f ine...continuing to 
parse...whoops! Youre trying to use a 
function before it's declared?!? Thafs NOT 
allowed around here... I'm outta here. 


Please wait.l 

Compiling 
your C++ 
code... 


®m\0 



Ah ha! The old "calling 
a function before youve defined 
it yet" trick, eh? I'll just make a note 
in case you def ine it later at runtime. 
You are planning to do that, right? 
Please don't disappoint me, or I'll give 
you an error... 


O 




Running yourl 
I Python code 
right now... 



Take another look at the error on the opposite 
page. Why do you think Python is giving you this 
particular error? What do you think is wrong? 



8S\ 
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line check 


Trace your code 

When you’re trying to work out what went wrong with a program that looks 
like it should be OK, a useful technique is to trace what happens as each line 
of code executes. Here’s the code that you are currently working with. At 
only three lines long (remember: the creation of the list is one line of code), it 
doesn’t look like it should cause any trouble: 



With the data assigned to the function’s arguments, 
to execute on each data item contained within the passed-in list: 


"To savc sy3£.c> the 
CfitifC C-ommc^t ,s 
*ot shoW*. 


LilC lUlIL lIUIl S L.UL1C old-I Lo 



. nc mov.es l.st is assijned to 
the_Jist , 3hd -the value O is 

assigned to “level" 


Prodess eath item ** 
-bV^c list... 

...then deeide what r - 
to do Kext based ok 
whethev ov hot the 
data item is a list- 


def print_lol (the__list, level) 
"""This function ... 



n n n 



for each_item in the_list: 

if isinstance (each__item, list) 

print_lol (each_item) -- 

else: 

for tab_stop in range(level) 
print("\t", end=’’) 
print(each_item) 


l-P the data item is a 
list re£u\rsively ‘mvoke 

the 'Puh£tioK...hahg ok 
a sttoy\d, that doesy/t 
look \righ t/? 
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Work out whaf $ wrong 


There 5 s your problem: the recursive invocation of your function is using the 
old function signature that required only one argument. The new version of 
your function requires two arguments. 

The fix is easy: provide the correct number of arguments when calling the 
new version of your function. So, this snippet of code from your function: 


if isinstance(each_item, list) 
print lol(each item) 


needs to be rewritten to specify the correct number of arguments: 


if isinstance(each_item, list): 
print lol(each item, level) 




Not so fast. Surely the nested 
list needs to be printed after a specif ic 
number of tab-stops? At the moment, your 
code sets "level" to 0 but never changes the 
value, so "level" never has any impact on your 
displayed output... 


Right. Your use of “level” needs one final tweak. 

The whole point of having level as an argument is to allow you to 
control the nested output. Each time you process a nested list, you need 
to increase the value of level by 1. Your code snippet needs to look 
like this: 


if isinstance(each_item, list): 
print lol(each item, level+1) 



lt’s time to perform that update. 


Simflv ■mtvemen-t t*»e value ' eve ' W 1 eath 
iwe you «tu«Welv mvoke WW 
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fresh pypi 


Update PyPI with your wew code 

Go ahead and edit your nester.py module (in the ne ster folder) to 
invoke your function properly. Now that you have a new version of your 
module, iti a good idea to update the distribution that you uploaded to PyPI. 

With your code amended, therei also a small change needed to your 
distributioni setup . py program. You Ve changed your API, so adjust the 
value associated with version in setup . py. Leti move from version 1.0.0 
to 1.1.0: 



Just as you did when you created and uploaded your distribution, invoke the 
setup . py program within your distribution folder to perform the upload: 


P<w't y°<A jwst 

love -thosc 
W Z0O 0^ 
mcssa^es? 


File Edit Window Help UpioadAgain 


$ python3 setup.py sdist upload 
running sdist 
running check 

reading manifest file 'MANIFEST' 
creating nester-1.1.0 
making hard links in nester-1.1.0... 
hard linking nester.py -> nester-1.1.0 
hard linking setup.py -> nester-1.1.0 
Creating tar archive 

removing 'nester-1.1.0' (and everything under it) 
running upload 

Submitting dist/nester-1.1.0.tar.gz to http://pypi.python.org/pypi 
Server response (200): OK 
$ 


Your new distribution is now available on PyPI. 
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f 

Mark 


Laura 


Mark: Take a look at this, guys.. .the ne ster module has been 
updated on PyPI. 

Bob: Version 1.1.0... 

Laura: I wonder whats changed? 

Mark: It stili works with nested lists of lists, but now you can see the 
nested structure on screen, which I think is pretty cool. 

Laura: And useful. Fve been waiting for that feature. 

Bob: Eh.. .OK.. .but how do I upgrade my existing local copy? 

Mark: Just follow the same steps as when you downloaded and 
installed ne ster from PyPI the first time. 

Bob: So I download the package file, unpack it, and ask setup . py to 
install it into my Python for me? 

Mark: Yes. It coukhft be any easier. 

Laura: And what about my existing version of ne ster; what 
happens to that “old” version? 

Bob: Yeah...do I have two ne ster modules now? 

Mark: No. When you use setup . py to install the latest version 
it becomes the current version and effectively replaces the previous 
module, which was the 1.0.0 release. 

Bob: And PyPI knows to give you the latest version of the module, too, 
right? 


Mark: Yes, when you surf the PyPI website and search for ne ster, 
you are always provided with the latest version of the module. 


Laura: Well, I use this module all the time and Fve been waiting for 
this feature. I think Fll update right away. 


Mark: Fve already upgraded mine, and it works a treat. 


Bob: Yeah, I use it a lot, too, so I guess Fll keep my system up to date 
and install the latest version. IFs probably not a good idea to rely on 
out-of-date Software, right? 


Mark: Fd say. And, there’s nothing quite like progress. 


Laura: Gatch you later, guys, Fve got work to do. 


Bob: Me, too. Fm off to PyPI to grab the latest ne ster and install 
it into my local copy of Python. Fll give it a quick test to confirm all is 


OK. 


Mark: Later, dudes... 
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unhappy user 


YouVe chawged your API 

Your new version of ne ster is indeed better, but not for all your users. 


Ah, phooey! I can'+ believe it... I installed 
the latest version of "nester" f rom PyPI, and 
now all of my code that uses your function is 
not working. What did you do?!? 



In your rush to release the lates and greatest version of your module, you 
forgot about some of your existing users. Recall that not all of your users want 
the new nested printing feature. However, by adding the second argument 
to print_lol () , you Ve changed your function 5 s signature, which means 
your module has a different API. Anyone using the old API is going to have 
problems. 



The ideal solution would be to provide both APIs, one which switches on the 
new feature and another that doesn’t. Maybe the feature could be optional? 


But how would that work? 
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Use optional arguments 


To turn a required argument to a function into an optional argument, provide 
the argument with a default value. When no argument value is provided, the 
default value is used. When an argument value is provided, it is used instead 
of the default. The key point is, of course, that the default value for the 
argument effectively makes the argument optional. 


To provide a default value for any function argument, specify the default 
value after the argument name: 



def print lol(the list, level=0) 


a,r 9 u ^Ch*ts 

** RBQUIRBt). 



The additio» 3 
de-fault value has 
■Uvned “levet 'mto 3» 
OPTIONAL arment- 


With the default value for the argument defined, you can now invoke the 
function in a number of different ways: 


jnvokc “the *funfct»on 
provide botb av^umehts- 


I^oke the Wtioh with 
both av-gumehts, but 
Pavide ah alterhative 
stavtihg value the 
se^ohd avgumeht. 


nester.print lol(movies, 0) 


nester.print lol(movies) 



nester.print lol(movies, 2) 


/hvoke the Wtion with 
ohe avgumeht ahd use 
the delrauli value -fov 
the sttor\A> 


Your function now supports different signatures, but the 
functonality remains as it was. 
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/ An IDLE Session- 

r 

Amend your code to give the level argument a default value of 0 and then load your code into the IDLE editor. 
Press F5 to load the code into the shell and then follow along to confirm that the latest version of your function 
works as expected. Start be defining a short list of lists and use the function to display the the list on screen: 

»> names = [' John' , ' Eric ' , [' Cleese' , ' Idle ' ] , ' Michael ' , [' Palin' ] ] 

»> print lol (names, 0) 


The sta*dard behavior works 

as C *P **ted f With hes-ted 

lists ihdehted. 


John 


Eric 



Cleese 


Idle 

Michael 



Palin 


Now try to do the same thing without specifiying the second argument. Let's rely on the default value kicking in: 
»> print_lol (names) 


John 


Eric 



Cleese 


Idle 

Michael 



Palin 



VV.tV.ou-t sfetity»* ite sedo*d tte 

de-fault is used a*d works, too. 


Now specify a value for the second argument and note the change in the functioni behavior: 
»> print_lol (names, 2) 

John 
Eric 


Michael 



Specfy a» altev-*ative value tov the sedohd 
^ and ^ Wewbnj stavts & 


*rom 


One final example provides what looks like a silly value for the second argument. Look what happens: 

»> print_lol (names, -9) 

Eric 1 Usm S a ne^atWe value ^ W* This looks e*adtly 

Cleese / t0 J Jf* W * ^ 

Idle | <Sr - |*e the crijM out ? ut U 1 .0.0, «*Wt. 

Michael / 

Palin / 
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Your module supports both APIs 

Well done! It looks like your module is working well, as both APIs, the original 
1.0.0 API and the newer 1.1.0 API, can now be used. 

Let’s take a moment to create and upload a new distibution for PyPI. As 
before, let’s amend the ver sion setting in the setup . py program: 



And with the code changes applied, upload this new version of your 
distribution to PyPI: 


This all looks 
«fine and 
dandy- 



File Edit Window Help UpioadThree 


$ python3 setup.py sdist upload 
running sdist 
running check 

reading manifest file 'MANIFEST' 
creating nester-1.2.0 
making hard links in nester-1.2.0... 
hard linking nester.py -> nester-1.2.0 
hard linking setup.py -> nester-1.2.0 
Creating tar archive 

removing 'nester-1.2.0' (and everything under it) 
running upload 

Submitting dist/nester-1.2.0.tar.gz to http://pypi.python.org/pypi 
Server response (200): OK 
$ 


Success! The messages from setup . py confirm that the your latest version 
of ne ster is up on PyPI. Let’s hope this one satisfies all of your users. 








Consider your code carefully. How might some of 
your users stili have a problem with this version of 
your code? 


you are here ► 


65 












faulty default 


Your API is stili wot right 



Although the API lets your users invoke the function in its original form, the 
nesting is switched on by default. This behavior is not required by everyone and 
some people aren’t at all happy. 


Another > 

version of "nester" has 
been released...but its 
default behavior might not 
be what you want. r 


I eant believe it! My 
programs were back to running 
fine, but now everything is 
indented. Has this thing 
changed again?!? ^ 


Of course, if you have some functionality that really ought to be optional 
(that is, not the default), you should adjust your code to make it so. But how? 

One solution is to add a third argument which is set to True when the 
indenting is required and False otherwise. If you ensure that this argument 
is False by default, the original functonality becomes the default behavior 
and users of your code have to request the new indenting feature explicitly. 


Let’s look at adding this final revision. 
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O Amend your module one last time to add a third argument to your function. Call your argument 
indent and set it initially to the value False— that is, do not switch on indentation by default. 
In the body of your function, use the value of indent to control your indentation code. 

Note: to save a bit of space, the comments from the module are not shown here. Of course, you 
need to make the necessary adjustments to your comments to keep them in sync with your code. 


def print lol(the list. 


Put "the extra av-gumcht 

here. 

i/ 

, level=0): 


for each_item in the_list: 

if isinstance(each_item, list): 

print_lol(each_item, 
else: 





, level+1) 


ftdd a Vme tode 

■to toirfbr 0 ^ ' w ^' er ' 



for tab_stop in 
print("\t", 


print(each item) 


range(level): 
end=’’) 



With your new code additions in place, provide the edit you would recommend making to the 
setup. py program prior to uploading this latest version of your module to PyPI: 



Provide the command you would use to upload your new distribution to PyPI: 
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adding an argument 



ExwctSe 


§ 0 (.|#tl 0 H 


O You were to amend your module one last time to add a third argument to your function. You were 
to call your argument indent and set it initially to the value False— that is, do not switch on 
indentation by default. In the body of your function, you were to use the value of indent to 
control your indentation code. 


Did you ihelude the 
de-fault value? 

_ / 

def print_lol (the_list, indewt—f fllse , level=0) : 

t» ^ 

for each_item in the_list: so uC sut 

lf isinstance(each_item, list): 

print_lol(each_item, indent , level + 1) 

else: 

" r ' der '‘t : < * he Colcm tte e«d of the "i-f" |i he . 

for tab stop in range(level): 

\\st Wk- print (" \t ", end=' ’ ) 

print(each_item) 

n sweet altemative to this M &v w | ooP 
,s this tode: * level, end—0. 


/* With your new code additions in place, you were to provide the edit you would recommend 
w making to the setup. py program prior to uploading this latest version of your module to PyPI: 


Edit w setup.py” so that it v-eads: vev-sion =■ i l3-0\ 

'CL Jt’ ! ‘ «w «m™ of »od„| e , so bt l„ ,l. . 

tt< «aW «Miafed w i, ^ vtpftil,. 

^ You were to provide the command you would use to upload your new distribution to PyPI: 


pythoni setuppy sdisk upload 


rf 7» « - «W- - V*i**W *~** 

mstead of "pytho^ • 
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«8* / An (DLE Session 


■r 


A final test of the functonality should convince you that your module is now working exactly the way you and your 


users want it to. Let's start with the original, default behavior: 

>>> names = ['John', 'Eric', ['Cleese', 'Idle'], 'Michael', ['Palin']] 
>>> print_lol(names) 

John 



Eric 


Cleese 


Idle 


Michael 


Palin 


Next, turn on indentation by providing True as the second argument: 

>>> names = ['John', 'Eric', ['Cleese', 'Idle'], 'Michael', ['Palin']] 
>>> print_lol(names, True) 

John 



Eric 


Cleese 


Idle 


Michael J 

Palin 

And, finally, control where indentation begins by providing a third argument value: 

>>> names = ['John', 'Eric', ['Cleese', 'Idle'], 'Michael', ['Palin']] 
>>> print lol(names, True, 4) 


John 



Eric 


Cleese 


Idle 


Michael 


Palin 




Go ahead and edit your 
setup . py file; then upload 
your distribution to PyPI. 
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one module for all 


Your modulet reputatiow is restored 

Gongratulations! Word of your new and improved module is spreading fast. 



This is as dosc as Bob ( 

■to a smile. Bu-t in,sl u , 
bc^s happy. © 


Your Python skills are starting to build 

YouVe created a useful module, made it shareable, and uploaded it to the 
PyPI website. Programmers all over the world are downloading and using 
your code in their projects. 


Keep up the good work. 
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Your Pythow Toolbox 

You’ve got Chapter 2 under your 
belt and you’ve added some key 


Python goodies to your toolbox. 



BULLET POINTS 


■ A module is a text file that contains Python 
code. 


Python L-m^o 

@ Use a 'V. ? le-<\uoied ^ 

a wJ-tifle-lme i* yowr todt- 

9 "PyPl" is tKe Py^o» P atka 5 e t" de * a " d 

*»s v/cll v/orth 3 Visit 

, A \awes^ate" is a flate m Pytho*»’* 

memov-y vjhev-e naw.es e*<st 
9 Python s m3in nawesfate is knovrn as 

main 


fDL-E Moie: 


*'T F? *> ■"»" C «i e i. tke IPU 
edit wihdow. 

■ ^,y° u P«« to "load" a modules 
dode the IDLE shell, the module's 
names ane sfeti^ieally imforted ^ 

™ S harn ! s P a£e ‘ This is a tonveniente 
When using IDL£ Mthin you, eode, you 

«eed to use the imponi statement 
expli£*rtly. 


■ The distributiori Utilities let you turn your 
module into a shareable package. 

■ The setup. py program provides 
metadata about your module and is used 
to build, install, and upload your packaged 
distribution. 

■ Import your module into other programs 
using the import statement. 

■ Each module in Python provides its own 
namespace, and the namespace name 
is used to qualify the module’s functions 
when invoking them using the module. 
function ( ) form. 

■ Specifically import a function from a module 
into the current namespace using the from 
module import function form of 
the import statement. 

■ Use # to comment-out a line of code or 
to add a short, one-line comment to your 
program. 

■ The built-in functions (BIFs) have their own 

namespace called_ builtins_, 

which is automatically included in every 
Python program. 

■ The range () BIF can be used with for to 
iterate a fixed number of times. 

■ Including end=' ' as a argument to the 
print () BIF switches off its automatic 
inclusion of a new-line on output. 

■ Arguments to your functions are optional it 
you provide them with a default value. 
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3 81es and excepfions 


X Dealing with errors + 



lt’s simply not enough to process your list data in your code. 

You need to be able to get your data into your programs with ease, too. It’s no surprise 
then that Python makes reading data from files easy. Which is great, until you consider 
what can go wrong when interacting with data externa! \o your programs...and there 
are lots of things waiting to trip you up! When bad stuff happens, you need a strategy for 
getting out of trouble, and one such strategy is to deal with any exceptional situations 
using Python’s exception handling mechanism showcased in this chapter. 


this is a new chapter 
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getting data in 


Pata is externaI to your program 

Most of your programs conform to the input-process-output model: data comes 
in, gets manipulated, and then is stored, displayed, printed, or transferred. 



pa-ta towes 

\o-ts 



So far, youVe learned how to process data as well as display it on screen. 
But what’s involved in getting data into your programs? Specifically, what 5 s 
involved in reading data from a file? 


How does Python read data from a file? 
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files and exceptions 


lt's ali lines of text 

The basic input mechanism in Python is line based: when read into your 
program from a text file, data arrives one line at a time. 

Python’s open () BIF lives to interact with files. When combined with a for 
statement, reading files is straightforward. 


Your dctt^ ^ w 

Jfilc talled w ske*Wvt*l • 





Y^ur da-fca 
as ihdividual 

lihcs. 


When you use the open () BIF to access your data in a file, an iterator is 
created to feed the lines of data from your file to your code one line at a time. 
But lef s not get ahead of ourselves. For now, consider the Standard open- 
process-close code in Python: 





Create a folder called 
HeadFirstPython and a 
subfolder called chapter3. 
With the folders ready, 
download sketch.txt from 
the Head First Python support 
website and save it to the 
ch apter 3 folder. 


Let’s use IDLE to get a feel for Python’s file-input mechanisms. 
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idle session 


/ An IDLE Session 


/ 

•r 

Start a new IDLE sesson and import the os module to change the current working directory to the folder that 
contains your just-downloaded data file: 

»> import os <_l*» ? ort "<*" U the Standard 

»> os . getcwd () lA/hat S the duVVent wovking divedtovy? 




Cbange to the folder that £ontains youv data -Pile. 


' /Users/barryp/Documents' J 

»> os . chdir (' . . /HeadFirstPython/chapter3 ') 

»> os. getcwd () \ ^— ConPivm you ave now in the vight plade- 

' /Users/barryp/HeadFirstPython/chapter3 ' J 


Now, open your data file and read the first two lines from the file, displaying them on screen: 

»> data = openCsketch.txt' ) ' Of>Cn a *>a**.ed -f ile a*>d assig*, -the -file to a -file objedt dalled "data". 

»> print (data, readline () , end=' ') 

Man: Is this the right room for an argument? 

»> print (data, readline () , end=' ') 

Other Man: I've told you once. 



Use the “veadlmeO" method to grab a 
Ime fro«* the f ile, the** use the >'mtO 
BIF to display it on sdveen. 


Let's "rewind" the file back to the start, then use a for statement to process every line in the file: 

Use the "seekO" method to retur** to the start of the f ile- 
/M yes, you da** use "tellO” *with Pytho**'s files, too. 


»> data.seek(O) 

0 

»> for each_line in data: 

print(each line, end='') 



Th.s dode should look familiar.- it's a sta**dard 


iteratio*, usmg the files data as m P ut- 


Fvevy line of the data is 
displayed on seveen (although 
for spate veasons, it is 
abvidged heve). 


Man: Yes it is! 
»> data. close () 


Sin£e you ave now done with the Pile, be suve to dose it- 
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Take a closer look at the data 


Look closely at the data. It appears to conform to a specific format: 

$ dolon, 'Collov/ed by a 
spadc dbav-adicv 

/ _ 

ManftIs this the right room for an argument? 


The tast 
members vole 


The line spoken by the 
easfc membev 



Other Man: I've told you once. 
Man: No you haven ' t! 

Other Man: Yes I have. 

Man: When? 

Other Man: Just now. 

Man: No you didn't! 



With this format in mind, you can process each line to extrnct parts of the line 
as required. The split () method can help here: 


i 1 J-U* "svlVtO” ««ethod M 

|nV ° Itedi Jth the "eath_W 

and bveak the *** 
and w 


Yihenevev a 



The split () method returns a list of strings, which are assigned to a list of 
target identifiers. This is known as multiple assignment : 
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idle session 




o 
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Kkow your data 

Your code worked fine for a while, then crashed with a runtime error. The 
problem occurred right after the line of data that had the Man saying, ‘You 
most certainly did not!” 

Let 5 s look at the data file and see what comes after this successfully processed 
line: 



Notice anything? 

Notice anything about the next line of data? 


The next line of data has two colons, not one. This is enough extra data 
to upset the split () method due to the fact that, as your code currently 
stands, split () expects to break the line into two parts, assigning each to 
role and line_spoken, respectively. 

When an extra colon appears in the data, the split () method breaks the 
line into threeparts. Your code hasn’t told split () what to do with the third 
part, so the Python interpreter raises aValueError, complains that you 
have “too many values,” and terminates. A runtime error has occurred. 





What approach might you take to solve this data- 
Processing problem? 



To help diagnose this 
problem, let 5 s put your 
code into its own file called 
sketch.py. You can copy 
and paste your code from 
the IDLE shell into a new 
IDLE edit window. 
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ask for help 


Know your wethods and ask for help 


It might be useful to see if the split () method includes any functionality 
that might help here. You can ask the IDLE shell to teli you more about the 
split () method by using the help () BIF. 




split(beans) 





The optional argument to split () Controls how many breaks occur within 
your line of data. By default, the data is broken into as many parts as is 
possible. But you need only two parts: the name of the character and the line 
he spoke. 

If you set this optional argument to 1, your line of data is only ever broken 
into two pieces, effectively negating the effect of any extra colon on any line. 

Let’s try this and see what happens. 



IDLE gives you searchable access to the entire Python 
documentation set via its Help —► Python Docs menu option (which 
will open the docs in your web browser). If all you need to see is the 
documentation associated with a single method orfunction, use 
the help () BIF within IDLE's shell. 




/\ 





► 
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— tf* / An 1DLE Session 


Here's the code in the IDLE edit window. Note the extra argument to the split () method. 
* n n sketch.py - /Users/barryp/HeadFirstPython/chapter4/sketch.py 


data = open( ’sketch.txt ') 

for each line in data: 

(role, line spoken) = eachline.split(' 
print(role, end= r ) 
print(’ said: ' , end='') 
print (line spoken, end= ) 

data.close() 



The extra argument 
Controls how "splitO" 
splits. 


Ln: 11 Coi: 0 


A 


With the edit applied and saved, press F5 (or select Run Module from IDLE's Run menu) to try out this version of 
your code: 


»> 

»> 


RESTART 


Man said: Is this the right room for an argument? 

Other Man said: I've told you once. 

Man said: No you haven't! 

Other Man said: Yes I have. -|-y^ (Jis^laycd ou*bfu*t is 

Man said: When? abridged to allow the 

Other Man said: Just now. important stwtt to -f it on 

this page 

Other Man said: Anyway, I did. 

Man said: You most certainly did not! 

Other Man said: Now let's get one thing quite ciear: I most definitely told you! 

Man said: Oh no you didn't! 

Other Man said: Oh yes I did! 

Man said: Oh no you didn't! 

Other Man said: Oh yes I did! ..Wt youV joy is shoY“t l»vcd* Theres 

Man said: Oh look, this isn't an argument! — /\ftOTH£R ValueError 1 , 


Traceback (most recent call last): 

File M /Users/barryp/HeadFirstPython/chapter4/sketch.py M , line 5, in <module> 
(role, line_spoken) = each_line.split(':', 1) 

ValueError: need more than 1 value to unpack 


Cool. y 0 u made it past the 

lihC “ty/o £olohS... 



That’s enough to ruin your day. What could be wrong now? 
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missing colon 


Kwow your data (befter) 


Your code has raised another ValueError, but this time, instead of 
complaining that there are “too many values,” the Python interpreter is 
complaining that it doesn’t have enough data to work with: “need more than 
1 value to unpack.” Hopefully, another quick look at the data will ciear up the 
mystery of the missing data. 


Other Man: Now let's get one thing quite ciear: I most definitely told you 
Man: Oh no you didn't! 

Other Man: Oh yes I did! 

Man: Oh no you didn't! 

Other Man: Oh yes I did! 

Man: Oh look, this isn't an argument! 

(pause) ^ 

Other Man: Yes it is! WhaCs this?/? Q otne ^ , , , 

Man: No it isn't! e ^ied tor»,at. .whidh ta*i be good- ™ 

(pause) ^ 

Man: It's just contradictioni 
Other Man: No it isn't! 



The case of the missing colon 


Some of the lines of data contain no colon, which causes a problem when 
the split () method goes looking for it. The lack of a colon prevents 
split () from doing its job, causes the runtime error, which then results in 
the complaint that the interpreter needs “more than 1 value.” 



0 
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files and exceptions 


Two very different approaches 



Or you could decide to let the 
errors occur, then simply handle 
each error if and when it happens. 
That would be exceptional. 


When you have to deal with a 
bunch of exceptional situations, the 
best approach is to add extra logic. 
If theres more stuff to worry 
about, you need more code. 


JiU’ s suggested approach certainly works: add the extra logic required to work out 
whether it’s worth invoking split () on the line of data. Ali you need to do is 
work out how to check the line of data. 

Joe 5 s approach works, too: let the error occur, spot that it has happened, and then 
recover from the runtime error.. .somehow. 


Which approach works best here? 
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find the substring 


Add extra logic 


Let’s try each approach, then decide which works best here. 

In addition to the split () method, every Python string has the find () 
method, too. You can ask find () to try and locate a substring in another 
string, and if it cani be found, the find () method returns the value -1. If 
the method locates the substring, find () returns the index position of the 
substring in the string 


i— J An IDLE Session 


7 

Assign a string to the each_line variable that does not contain a colon, and then use the find () method to 
try and locate a colon: 

»> each_line = "I teli you, there's no such thing as a flying circus." 

»> each_line. find (' : ') 

1 _- The stvina does NOT do*iai* a dolo*, so "tmdO" veiuvins -I -Cov NOT FOWNP- 


•1*9 

Press Alt-P twice to recall the line of code that assigns the string to the variable, but this time edit the string to 
include a colon, then use the find () method to try to locate the colon: 

»> each_line = "I teli you7\there's no such thing as a flying circus." 

»> each line. find (' : T ) 


10 



The s-bring D0B£ doirtam a dolon, so w -PindO w v-etuv-ns a posi-tive index value. 


And you thought this approach wouldht 
work? Based on this IDLE session, I 
think this could do the trick. 


O 


o 
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files and exceptions 



Adjust your code to use the extra logic technique demonstrated on the previous page to deal 
with lines that don’t contain a colon character. 


data = open('sketch.txt') 

for each_line in data: 
if 

(role, line_spoken) = each_line.split(':', 1) 
print(role, end='') 
print(' said: \ end='') 
print(line spoken, end='') 


VVV»at do*d'rt'»o* nccds to 50 ^«vc? 



data.close () 



your pencil 


Can you think of any potential problems with this technique? 
Grab your pencil and write down any issues you might have with 
this approach in the space provided below: 
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substring found 



ExwctSe 


§0(.|#tl0H 


You were to adjust your code to use the extra logic technique to deal with lines that don’t contain 
a colon character: 


data = open('sketch.txt') 


for each line in data: 



Ii iakes a £ew setonds io get your 
head airouhd this aditio*, but it 
does work. 


if 



y \ o ’t eadh^Jme-fmd^O -l ; 

(role, line_spoken) = each_line.split(' 
print(role, end='') 
print(' said: \ end='') 
print(line spoken, end='') 


1) 


data.close () 



1-t’s OK 1 Y wr i ss , es 
ave ^ so 

LL-vi SUnMld^ 


\nv\0t 


You were to think of any potential problems with this technique, 
grabbing your pencil to write down any issues you might have 
with this approach. 


{fi these- 

There mitjht be a problem v/ith this dode i-f the -format o-f the data -file 
dhawjes; v/hidh v/ill require dhar^es to the dor\ditior\. 



The dor\ditior\ used by the i-f statement is somev/hat hard to read a*d 
ur\derstar\d- 


This dode is a little u -fra<jile”...it v/ill break i-f another e%deptior\al 
situatior\ arises. 
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files and exceptions 



Amend your code within IDLE's edit window, and press F5 to see if it works. 




Python Shell 


>» ====== 

»> 

Man said: 
Other Man 
Man said: 
Other Man 
Man said: 
Other Man 
Man saidi 
Other Man 
Man said; 
Other Man 
Man said; 
Other Man 
Man said; 
Other Man 
Other Man 
Man said; 
Other Man 
Man said; 
Other Man 
Man said; 
Other Man 
Man said; 
Other Man 
Man said; 
Man said: 
Other Man 
Man said; 
Other Man 
Man said; 
Other Man 
Man said; 
Other Man 
Man said; 
Other Man 
Man said; 
Other Man 
Man said: 
»> I 


RESTART 


the right room for 
ve told you once. 

1I 


Is this 
said; I 
No you haven 
said; Yes I have. 

When? 

said: Just now. 

No you didn't! 
said: Yes I didi 

You didn ' tI 

said: I r m telling you, I didi 

You did noti 
said; Oh I F m sorry, 

AhI (taking out his 
said: Just the five 

said: Anyway, I did, 

You most certainly did 


an argument? 


Now 

you 


said; 

Oh no 
said: Oh yes I 

Oh no you didn 
said: Oh yes I 

Oh look, this 


let F s get 
didn r 11 

didi 
F tl 
didi 
isn 1 1 


is this a five minute argument, or the full half hour? 
wallet and paying) Just the five minutes, 
minutes, Thank you. 

noti 

one thing quite ciear: I most definitely told youl 


an argumenti 


is i 


said: Yes it 

No it isn F 11 

It 1 s just contradictioni 
said: No it isn F tI 

It ISI 

said: It is NOTI 

You just contradicted mei 
said; No I didn F tI 
You DIDI 

said: No no nol 
You did just thenl 
said: NonsenseI 
(exasperated) Oh, this 
said: No it isn F tI 

Yes it is i 




No eVToirs 

iime. 


is futilelI 


Lm: 


149 Coi: 


A 


Your program works... although it is fragile. 

If the format of the file changes, your code will 
need to change, too, and more code generally means 
more complexity. Adding extra logic to handle 
exceptional situations works, but it might cost you 
in the long run. 
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exceptional catch 


Handle exceptions 


Have you noticed that when something goes wrong with your code, the 
Python interpreter displays a traceback followed by an error message? 

The traceback is Python 5 s way of telling you that something unexpected has 
occurred during runtime. In the Python world, runtime errors are called 

exceptions. 


0 


Whoooah! I don'+ 
know what to do with this 
error, so I'm gonna raise 
an excep+ion...this really is 
someone elses problem. 


, u l u \ooVs 

ooov» Y*; ■ 

\\V* tVffre s 3 * 



»> if not each 

Traceback (most rj 
File "<pyshell 
(role, line_ 
ValueError: too mP 




Of course, if you decide to ignore an exception when it occurs, your program 
crashes and burns. 

But here 5 s the skinny: Python let’s you catch exceptions as they occur, which 
gives you with a chance to possibly recover from the error and, critically, not 
crash. 

By controlling the runtime behavior of your program, you can ensure (as 
much as possible) that your Python programs are robust in the face of most 
runtime errors. 


Try the code first. Then deal with errors as they happen. 
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files and exceptions 


Try first, thew recover 


Rather than adding extra code and logic to guard against bad things 
happening, Python 5 s exception handling mechanism lets the error occur, 
spots that it has happened, and then gives you an opportunity to recover. 

During the normal jlow of control , Python tries your code and, if nothing goes 
wrong, your code continues as normal. During the exceptional jlow of control , 
Python tries your code only to have something go wrong, your recovery code 
executes, and then your code continues as normal. 







Yowv ve6ovevy 
tode e*edwtes 


The try/except mechanism 

Python includes the try statement, which exists to provide you with a way to 
systematically handle exceptions and errors at runtime. The general form of 
the try statement looks like this: 




Crask! 



Yowv e*teption 
is handled- 


Theh you keep 
3°ih 9 ... 



BotV» "tv-y" u 
and Vdeft" 
are Python 
keyxords- 
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allow errors 



Hang on, here! Are you 
actually letting errors 
occur on purpose? ARE 
YOU MAD?!? 


O 


No. Not mad. And, yes. Letting errors occur. 

If you try to code for every possible error, you’11 be at it for a 
long time, because all that extra logic takes a while to work out. 

Paradoxically, when you worry less about covering every 
possible error condition, your coding task actually gets easier. 
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files and exceptions 


Idcwtify the code to protect 


In order to plug into the Python exception handling mechanism, take a 
moment to identify the code that you need to protect. 


, r 


Study your program and circle the line or lines of code that you 
think you need to protect. Then, in the space provided, state why. 


data = open('sketch.txt') 


for each_line in data: 

(role, line_spoken) = each_line.split(' : ' , 1) 

print(role, end='') 
print ( ' said: ' r end='') 
print(line spoken, end='') 


data.close () 

State ^ ’ reas<m 



t[iere i ure no 

Dumb Questions 


NL* Something has been bugging me for a while. When the split() method executes, it passes back a list, but the target identifiers 
are enclosed in regular brackets, not square brackets, so how is this a list? 

Well spotted. It turns out that there are two types of list in Python: those that can change (enclosed in square brackets) and those that 
cannot be changed once they have been created (enclosed in regular brackets). The latter is an immutable list, more commonly referred 
to as a tuple. Think of tuples as the same as a list, except for one thing: once created, the data they hold cannot be changed under any 
circumstances. Another way to think about tuples is to consider them to be a constant list. At Head First, we pronounce “tuple” to rhyme with 
“couple.” Others pronounce “tuple” to rhyme with “rupal.” There is no ciear concensus as to which is correct, so pick one and stick to it. 
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code to protect 


r r:\rr 



Solutiori 


You were to study your program and circle the line or lines of 
code that you think you need to protect. Then, in the space 
provided, you were to state why. 


data = open(’sketch.txt’) 



TV*se W 

Vmes o£ tode 
al\ *eed to 


|*P *the tali *to w splrtO w -Pails, you dor/*t u/an*t *the *th\ree M prirrfcO w 
s*ta*tcinr\cir>*ts e%etu*t‘mg>, so i^s best *fco pv-o-tcdt all -Pour lines o-P *tbe U i-P 
suite, not just *tbe Ime o-P tode *tba*t talis w sj>li*tO w . 




OK. I get that the code 
can be protected f rom an 
error. But what do I do when 
an error actually occurs? 
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files and exceptions 


Take a pass on the error 

With this data (and this program), it is best if you ignore lines that don’t 
conform to the expected format. If the call to the split () method causes 
an exception, let’s simply pass on reporting it as an error. 

When you have a situation where you might be expected to provide code, but 
don’t need to, use Python’s pass statement (which you can think of as the 
empty or null statement.) 

Here 5 s the pass statement combined with try: 



Now, no matter what happens when the split () method is invoked, the 
try statement catches any and all exceptions and handles them by ignoring the 
error with pass. 

Let’s see this code in action. 



Make the required changes 
to your code in the IDLE 
edit window. 
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idle session 
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files and exceptions 


Fireside Chats 



Tonighfs talk: Approaching runtime errors with extra code 
and exception handlers 


Extra Code: Exception Handler: 

By making sure runtime errors never happen, I keep 
my code safe from tracebacks. 

At the cost of added complexity.... 

Complexity never hurt anyone. 

FU be sure to remind you of that the next time 
you’re debugging a complex piece of code at 4 
oYlock in the morning. 

I just don’t get it. You’re more than happy for your 
code to explode in your face.. .then you decide it 5 s 
probably a good idea to put out the fire?!? 

Yes. I concentrate on getting my work done first and 
foremost. If bad things happen, I’m ready for them. 

But the bad things stili happen to you. They never 

happen with me, because I don’t let them. T T • i . . . . . , 

Until somethmg else happens that you weren t 

expecting. Then you’re toast. 

Well...that depends. If you’re smart enough—and, 
believe me, I am—you can think up all the possible 
runtime problems and code around them. 

Sounds like a whole heap of extra work to me. 

Hard work never hurt anyone. 

You did hear me earlier about debugging at 4 AM, 
right? Sometimes I think you actually enjoy writing 
code that you don’t need... 

Of course all my code is needed! How else can you 
code around all the runtime errors that are going to 
happen? 

Yeah.. .how many? 

Um, uh.. .most of them, I guess. 

You don’t know, do you? You Ve no idea what will 
happen when an unknown or unexpected runtime 
error occurs, do you? 

Look: just cut it out. OK? 
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more errors 


What about other errors? 

It is true that both approaches work, but let’s consider what happens when 
other errors surface. 



Handliwg wissiwg files 

Frank’s posed an interesting question and, sure 
enough, the problem caused by the removal of 
the data file makes life more complex for Jill and 
Joe. When the data file is missing, both versions of 
the program crash with an IOError. 




Rename the data file, then 
run both versions of your 
program again to confirm 
that they do indeed raise an 
IOError and generate a 
traceback. 
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Add more error-checking code... 

If you’re a fan of the u let’s not let errors happen” school of thinking, your 
first reaction will be to add extra code to check to see if the data file exists 
before you try to open it, right? 

Let’s implement this strategy. Python’ s o s module has some facilities that can 
help determine whether a data file exists, so we need to import it from the 
Standard Library, then add the required check to the code: 


Chefck whe-thev 
{\\t -file 


import os 

if os.path.exists(’sketeh.txt’) 

data = open( 1 sketeh.txt') 


AII o( -this 

CoAc VCmdihS 


for each_line in data: 

if not each_line.find( 1 : 1 ) == -1: 

(role, line_spoken) = each_line.split(’ 
print(role, end= 11 ) 
print ( 1 said: ', end= 1 1 ) 
print(line spoken, end= 11 ) 


1) 


flvfoVlto -the 
wev- ot -the 
b ad hcws. 


data.close () 


else: 


print(’The data file is missing!’) 


* / An IDLE Session- 

A quick test of the code confirms that this new problem is dealt with properly. With this new version of your code 
in IDLE's edit window, press F5 to confirm all is OK. 


»> 


RESTART 


»> 

The data file is missing! ^ 

»> 


£*attly *Aat Coo I. 
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take it to another level 


...Or add another level of exception handliwg 

If you are a fan of the “handle exceptions as they occur” school of thinking, you’11 
simply wrap your code within another try statement. 


hdd ano-fcliev- 
s idicrneir\-fc 


yii*th “the 
o-thev pvoiyrar» 

all "tWis 
todt ver*ains 
ufidhav^cd* 


£jWe “the uscv 
-the bad v\e>ws- 




try: 

data = open( 1 sketch.txt 1 ) 

for each_line in data: 
try: 

(role, line_spoken) = each_line.split( 1 
print (role, end=' 1 ) 
print ( 1 said: end= 11 ) 

print(line_spoken, end='') 
except: 
pass 

data.close () 

except: 

print(’The data file is missing!’) 


1) 


•* / An IDLE Session- 

U-r 

Another quick test is required, this time with the version of your program that uses exception handling. Press F5 to 
give it a spin. 


»> 


»> 

The data file is missing! ^ 

»> 


RESTART ======================= 

As e*pefcted, tW.s vevsio» of 
?v . 0 ya» handles the missi^ t.le, W 


98 Chapter 3 



















files and exceptions 


$o, which approach is best? 

Well.. .it depends on who you ask! Here are both versions of your code: 


& ^ sketch-extra-logic-file-missing.py - /Users/barryp/HeadFirstPython/chapter3/sk... 


import os 

if os.path.exists( ' sketch.txt ’ ): 
data = open( 1 sketch.txt ' ) 


for each_line in data: 

if not each_line.find( ' : ’ ) == -1: 

(role, linespoken) = each_line.split( ' : ' , 1) 
print(role, end= '' ) 
print ( ' said ', end='') 
print (line_spoken, end='') 

^ ^ sketch-with-try-file-missing.py -/Users/barryp/HeadFirstPython/chapter3/sket.. 

data.close() 


This veirsioh uses exiv-a lo«id 
t» ha»dle File 1/0 errov-s. 


else: 

print(’The datafile is missingl ’ ) 


This ve«ioh uses a*other V 
stateme^t -to hahdle Fle 1/0 e 



CV“\ro\rs. 


try : 

data = open( ’ sketch.txt ' ) 

for each_line in data: 
try: 

(role, linespoken) = each_line.split( ' : ' , 1) 
print (role, end='') 
print ( ’ said ’, end= ’’ ) 
print (linespoken, end= '’ ) 

except : 
pass 

data.close() 

except : 

print(’The datafile is missingl') 


Ln: 17 Coi: 0 


Let 5 s ask a simple question about these two versions of your program: What do each 
of these programs do? 



your pencil 


Grab your pencil. In box 1, write down what you think the 
program on the left does. In box 2, write down what you think 
the program on the right does. 
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keep it simple 

^Jharpen your pencil_ 

\a|i if ion You were t0 9 ra b y° ur P enc if then in box 1, write down what you 

11 thought the program on the left does. In box 2, write down what 

, s a loi ^ ^ so Y ou adWlY »«<* you thought the program on the right does - 

r JZlfauU yo«r acstv. ? -bVo« tKa* '«as 

^vovided the frevious pay- -y 

or The dode on the \ri^ht sta^rts by imporbinc} the M os w libvavy, and *then it uses w pa*th.e%ists W to 
make sure the data -Pile e%is*ts, be-Pore it atterwpts to ope* the data -Pile. Eadh Ime -Prom 
the -Pile is the* prodessed, but only a-Pter it has detey-^med that the Ime doh-Po^r^s to the 
re<\uired -Pormat by dhedking -Piv-st -Por a sin^le w : w dhav-adtev- in the line- |*P the w i w is -Pound, 
the line is prodessed; othev-v/ise, it^s i^nored- l/Vhen u/eVe all done, the data -Pile is dlosed- /Ud 
you ^et a -Priendly messa^e at the end i-P the -Pile is not -Pound- 

NoY/...that’s more like it- 

The dode on the vight opens a data -Pile, prodesses eadh line in that -Pile, e*tradts the data o-P 
interest and displays it on sdreen- The -Pile is dlosed v/hen done- |*P any e^deptions oddur, this 
dode handles them. 


o 


Complexity is rarely a good thing 

Do you see what 5 s happening here? 

As the list of errors that you have to worry about grows, the complexity of 
the “add extra code and logic” solution increases to the point where it starts 
to obscure the actualpurpose of the program. 

This is not the case with the exceptions handling solution, in which it 5 s obvious 
what the main purpose of the program is. 

By using Python’s exception-handling mechanism, you get to concentrate on 
what your code needs to do, as opposed to worrying about what can go wrong 
and writing extra code to avoid runtime errors. 

Prudent use of the try statement leads to code that is easier to read, easier 
to write, and—perhaps most important—easier to fix when something goes 
wrong. 

Concentrate on what your code needs to do. 
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You re done...except for owe small thmg 

Your exception-handling code is good. In fact, your code might be too good in 
that it is too geneml. 

At the moment, no matter what error occurs at runtime, it is handled by your 
code because it’s ignored or a error message is displayed. But you really need to 
worry only about IOErrors and ValueErrors, because those are the types 
of exceptions that occurred earlier when your were developing your program. 

Although it is great to be able to handle all runtime errors, it’s probably 
unwise to be too generic.. .you will want to know if something other than 
an IOError or ValueError occurs as a resuit of your code executing at 
runtime. If something else does happen, your code might be handling it in an 
inappropriate way. 



As your code is currently written, it is too generic. Any runtime error that 
occurs is handled by one of the except suites. This is unlikely to be what 
you want, because this code has the potential to silently ignore runtime errors. 

You need to somehow use except in a less generic way. 
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specify exceptions 


Pe specific with your exceptions 

If your exception-handling code is designed to deal with a specific type of 
error, be sure to specify the error type on the except line. In doing so, you’11 
take your exception handling code from generic to specific. 



Of course, if an different type of runtime error occurs, it is no longer handled 
by your code, but at least now you’11 get to hear about it. When you are 
specific about the runtime errors your code handles, your programs no longer 
silently ignore some runtime errors. 








files and exceptions 



Your Pythow Toolbox 

You’ve got Chapter 3 under your 
belt and you’ve added some key 
Python techiques to your toolbox. 


,A 

, -.]/•’ \s a de-W'' e 

«atW' 


fl>LE Afefe. 

srf ? 

^ k f'" '» 




BULLET POINTS 


Use the open () BIF to open a disk file, 
creating an iterator that reads data from 
the file one line at a time. 

The readlineO method reads a 
single line from an opened file. 

The seek () method can be used to 
“rewind” a file to the beginning. 

The close () method closesa 
previously opened file. 

The split () method can break a 
string into a list of parts. 

An unchangeable, constant list in Python 
is called a tuple. Once list data is 
assigned to a tuple, it cannot be changed. 
Tuples are immutable. 

AvalueError occurs when your data 
does not conform to an expected format. 

An iOError occurs when your data 
cannot be accessed properly (e.g., 
perhaps your data file has been moved or 
renamed). 

The help () BIF provides access to 
Python’s documentation within the IDLE 
Shell. 

The f ind () method locates a specific 
substring within another string. 

The not keyword negates a condition. 

The try/except statement provides 
an exception-handling mechanism, 
allowing you to protect lines of code that 
might resuit in a runtime error. 

The pass statement is Python’s empty 
or null statement; it does nothing. 
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4 persistence 


* Saving data to files ^ 



It is truly great to be able to process your file-based data. 

But what happens to your data when you’re done? Of course, it’s best to save your 
data to a disk file, which allows you to use it again at some later date and time. Taking 
your memory-based data and storing it to disk is what persistence is all about. Python 
supports all the usual tools for writing to files and also provides some cool facilities for 
efficiently storing Python data. So...flip the page and let’s get started learning them. 


this is a new chapter 
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save your work 


Programs produce data 

It’s a rare program that reads data from a disk file, processes the data, and 
then throws away the processed data. Typically, programs save the data they 
process, display their output on screen, or transfer data over a network. 

have lo-fcs o-f choices 
aM; y/hiih ^ of aisk 
t ile io usc. 



Before you learn what’s involved in writing data to disk, let 5 s process the data 
from the previous chapter to work out who said what to whom. 

When that’s done, you’ll have something worth saving. 


106 Chapter 4 


















persistence 



Code Magnets 


Add the code magnets at the bottom of this page to your existing 
code to satisfy the following requirements: 


1. Create an empty list called man. 

2. Create an empty list called other. 

3. Add a line of code to remove unwanted whitespace from the 

line_spoken variable. 

4. Provide the conditions and code to add line_spoken to the 
correct list based on the value of role. 

5. Print each of the lists (man and other) to the screen. 


try: 

data = open('sketch.txt') 
for each_line in data: 
try: 

(role, line spoken) = each line.split(':', 1) 


except ValueError: 
pass 
data.close () 
except IOError: 

print('The datafile is missing!') 


tteve ave vouv ma^e-b- 
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process and print 



Code Magnets Solution 

Your were to add the code magnets to your existing code to satisfy 
thefollowing requirements: 

1. Create an empty list called man. 

2. Create an empty list called other. 

3. Add a line of code to remove unwanted whitespace from the 

line_spoken variable. 

4. Provide the conditions and code to add line_spoken to the 
correct list based on the value of role. 

5. Print each of the lists (man and other) to the screen. 



try: 


Assijjh 3h empty | IS £ to 
"**>" and ‘'other". 


data = open('sketch.txt') 
for each line in data: 



try: 


Assigh -the 
s-tv-ipped strihg 
badk oh"to itsel-p 


"eli-f” »»eahs 
"eUe 1" 


(role, line spoken) = each line.split(’ 




elif )role == 'Other Man': 


other.append(line_spoken) 


, 1) 



The n>e-thod 

removes uhwahted 

vjhitespade Pom a 


Upda-te OhC the lists 
b^sed oh who said what- 


except ValueError: 
pass 

data.close () 
except IOError: 

print('The datafile is missing!’) 


[ 


print(man) 


print(other) 


Cohtlwde by d.splaym°i ibe 

pvoeessedi data oh sdveeh. 
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persistence 



Load your code into IDLE’s edit window and take it for a spin by pressing F5. Be sure to save your program into the 
same folder that contains sketch. txt. 


^ ^ ' ' 2-open-into-lists-cleaned-up.py - /Users/barryp/HeadFirstPython/chapter4/2-o... 


man = [] 
other = [] 

try s 

data = open( 1 sketch.txt ’ ) 

for each line in data: 
try: 

(role, linespoken) = each_line.split( ’ : ’ , 1) 
linespoken = linespoken.strip() 
if role == ’ Man ’ : 

man.append(linespoken) 
elif role == 'Other Man': 

other.append(line spoken) 
except ValueError: 
pass 

data.close() 
except IOError: 

print(’The datafile is missingl ' ) 


The tode in |DLE's 
cd\x window 



print (man) 
print (other) 


« rs o 


Python Shell 


W hev* s viViat aleare 
tV>c kuo lists* 


X 7 




Python 
(GCC 4 
Type 
»> 

»> 

[ *ls 


3.1.2 (r312:79360M, Mar 24 2010, 01:33:18) 

0.1 (Apple Inc. build 5493)] on darwin 
"Copyright", "credits" or "license()" for more information. 
=============================== RESTART =================== 


this 


the right room for an argument?', "No you haven’tl", 'When?', "No yo 
u didn'tl", "You didn’tl", 'You did noti’, 'Ah! (taking out his wallet and pay 
ing) Just the five minutes.’, 'You most certainly did not!', "Oh no you didn't 
1", "Oh no you didn'tl", "Oh look, this isn’t an argumenti", "No it isn'ti", " 
It's just contradictioni", 'It IS!’, 'You just contradicted me!', 'You DID!', 
'You did just then!', '(exasperated) Oh, this is futile!!', 'Yes it is!'] 

("I’ve told you once.", 'Yes I have.’, 'Just now.', 'Yes I did!’, "I’m telling 
you, I didi", "Oh I’m sorry, is this a five minute argument, or the full half 
hour?", 'Just the five minutes. Thank you.', 'Anyway, I did.', "Now let's get 
one thing quite ciear: I most definitely told you!", 'Oh yes I did!’, 'Oh yes 
I did!', 'Yes it is!', "No it isn'tl", 'It is NOT!’, "No I didn'tl", 'No no no 
!’, 'Nonsense!', "No it isn’tl"] 

»> 


Ln: 8 Coi: 4 


It worked, as expected. 




Yes, it can. 

When you need to save data to a file, the 
open () BIF is all you need. 
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operi and close 


Open your file in write mode 


When you use the open () BIF to work with a disk file, you can specify an 
occess mode to use. By default, open () uses mode r for reading , so you don’t 
need to specify it. To open a file for writing , use mode w: 


The a££ess modeI 
io use 



By default, the print () BIF uses Standard output (usually the screen) when 
displaying data. To write data to a file instead, use the file argument to 
specify the data file object to use: 


print("Norwegian Blues stun easily.", file=out) 

T- 

Mat geis wiri-fcteh io ih e -fj| e 



- 


When you’re done, be sure to close the file to ensure all of your data is written 
to disk. This is known as flushing and is very important: 


The «a*«e ot ihe daia 

fle objei-i io 


out.close() 


1 

J 


This is VBRY importari 

'«heh wviiing io ti| e s. 



When you use access mode w, Python opens your named file 
for writing. If the file already exists, it is cleared ofits contents, or 
clobbered. To append to a file, use access mode a, and to open a 
file for writing and reading (without clobbering), use w+. If you 
try to open a file for writing that does not already exist, it is first 
created for you, and then opened for writing. 
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r Sharpen your pencil 


persistence 


At the bottom of your program, two calls to the print () BIF 
display your processed data on screen. Let's amend this code to 
save the data to two disk files instead. 


man 


[] 


Call your disk files man_data . txt (for what the man said) and 
other_data . txt (for what the other man said). Be sure to 
both open and close your data files, as well as protect your code 
against an IOError using try/except. 


other = 


try: 


data = open(’sketch.txt’) 
for each line in data: 


try: 

(role, line_spoken) = each_line.split( 1 : 1 , 1) 

line_spoken = line_spoken.strip() 
if role == ’Man’: 

man.append(line_spoken) 
elif role == 'Other Man’: 


other.append(line spoken) 


except ValueError: 
pass 
data.close () 
except IOError: 

print (’The datafile is missing!’) 



OpCh youv -fcy/o 
data -files heve. 




print(man. 


print(other. 
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save to disk 



man = [] 
other = [] 


At the bottom of your program, two calls to the print () BIF 
display your processed data on screen. You were to amend this 
code to save the data to two disk files instead. 

You were to call your disk files man_data . txt (for what the 
man said) and other_data . txt (for what the other man said). 
You were to make sure to both open and close your data files, as 
well as protect your code against an IOError using try/except. 


try: 


data = open('sketch.txt') 
for each_line in data: 
try: 


(role, line_spoken) = each_line.split(' 
line_spoken = line_spoken.strip() 
if role — 'Man': 

man.append(line_spoken) 
elif role == 'Other Man': 

other.append(line_spoken) 
except ValueError: 
pass 
data.close () 
except IOError: 

print('The datafile is missing!') 


1) 


AH o-f -this Lodt is 
un^hanged. 


brf 



mantile — openCman_data*My V) 
o-tbev^-file — openCotbev^datat*^; V) 


print (man, -file^man_-file ) 

print (other, 'file= 1 o'the\r -file ) 



pid YOU \remembcv -to open 
youv -Piles m W/RITfc mode? 

OpCh you\r -two -files, and assign 
eaeh -to -file objee-ts- 


Use the y-m-fcO" BIF io save ih e 
n * ned lis ts i» na*>ed disk 6| es . 


man_jfiledloseO 
o*tbev-_JfiledoseO 
e%dep-t lO&riror: 

prirrt^Pile evrov.O 



Pont foryt io tlose BOTH -files. 

Handle an |/0 exCeption, should 
one 
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persistence 



Perform the edits to your code to replace your two print () calls with your new file l/O code. Then, run your 
program to confirm that the data files are created: 


« n 


chapter4 


4 


nn 


o 




Back 


View 


Quick Look Aetion TermHere Touch 




Home 


Search 


DEVICES 

__ Macintosh HD 

Paul Barry's MacBook Pro 

PLACES 

Applications 
barryp 
frtl Desktop 
Boards 
J Compstaff 
o Downloads 


sketch.txt 


PYTHON 


sketch4.py 


«e 



Python Shell 


Python 3.1.2 (r312:79360M, Mar 24 2010, 01:33:18) 

[GCC 4.0.1 (Apple Inc. build 5493)] on darwin 

Type "Copyright", "credits" or "license()" for more information. 
»> ================================ RESTART =================== 

»> 

»> 


there a^e no data -files 
youir tolder, just youv tode- 




Ln: 6 Coi: 4 


y/ken yow rw» 
youv* todti tkis 

is all yow 

wkat looks like 

a» w e*»pfcy IPt£ 

skell- 


& ^ ^ chapter4 



That code worked, too. YouVe created two data files, each holding the data 
from each of your lists. Go ahead and open these files in your favorite editor 
to confirm that they contain the data you expect. 



WAkN 


Consider the following carefully: what happens to 
your data files if the second call to print () in 
your code causes an iOError? 
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c/ose files 


Files are left open after an exceptioni 

When all you ever do is read data from files, getting an IOError is annoying, 
but rarely dangerous, because your data is stili in your file, even though you 
might be having trouble getting at it. 

It 5 s a different story when writing data to files: if you need to handle an 
IOError before a file is closed, your written data might become corrupted 
and there 5 s no way of telling until after it has happened. 


try: *S(k. 

man_f ile = open (’ man_data. txt ’ , ’ w ’) 
other_f ile = open (’ other_data. txt 1 , ’ w ’) 

print(man, file=man_file) 
print (other, f ile=other_file) f^loi Otfl 



man_file.close() 
other_file.close() 
except IOError: 

print(’File error.’) 


Your exception-handling code is doing its job, but you now have a situation 
where your data could potentially be corrupted, which cafft be good. 

What 5 s needed here is something that lets you run some code regardless of 
whether an IOError has occured. In the context of your code, you’11 want 
to make sure the hies are closed no matter what. 
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persistence 


Extend try witb finally 

When you have a situation where code must always run no matter what errors 
occur, add that code to your try statemenfs f inally suite: 



If no runtime errors occur, any code in the f inally suite executes. Equally, 
if an IOError occurs, the except suite executes and then the f inally 
suite runs. 

No matter what, the code in the finally suite always runs. 

By moving your file closing code into your finally suite, you are reducing 
the possibility of data corruption errors. 

This is a big improvement, because you’re now ensuring that files are closed 
properly (even when write errors occur). 

But what about those errors? 

How do you find out the specifics of the error? 
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no dumb questiohns 


there icffe no 

Dumb QuestiQtis 


q l’m intrigued. When you stripped the line_spoken data 
of unwanted whitespace, you assigned the resuit back to the 
line_spoken variable. Surely invoking the strip() method on 
line_spoken changed the string it refers to? 

No, that’s not what happens. Strings in Python are immutable, 
which means that once a string is created, it cannot be changed. 

Q: But you did change the line_spoken string by removing 
any unwanted whitespace, right? 

Yes and no. What actually happens is that invoking the 
strip () method on the line_spoken string creates a 
new string with leading and trailing whitespace removed. The new 
string is then assigned to line_spoken, replacing the data that 
was referred to before. In effect, it is as if you changed line_ 
spoken, when youVe actually completely replaced the data it 
refers to. 


O: 


So what happens to the replaced data? 


-^1 Python’s built-in memory management technology reclaims the 
RAM it was using and makes it available to your program. That is, 
unless some other Python data object is also referring to the string. 


O: 


What? I don’t get it. 


It is conceivable that another data object is referring to the 
string referred to by line_spoken. For example, let’s assume 
you have some code that contains two variables that refer to the 
same string, namely “Flying Circus.” You then decide that one of 
the variables needs to be in ali UPPERCASE, so you invoke the 
upper () method on it. The Python interperter takes a copy of the 
string, converts it to uppercase, and returns it to you. You can then 
assign the uppercase data back to the variable that used to refer to 
the original data. 

Q: And the original data cannot change, because there’s 
another variable referring to it? 

Precisely. That’s why strings are immutable, because you never 
know what other variables are referring to any particular string. 


Q: But surely Python can work out how many variables are 
referring to any one particular string? 

It does, but only for the purposes of garbage collection. If you 
have a line of code like print (' Flying Circus ' ), the 
string is not referred to by a variable (so any variable reference 
counting that’s going on isn’t going to count it) but is stili a valid string 
object (which might be referred to by a variable) and it cannot have 
its data changed under any circumstances. 

Q: So Python variables don’t actually contain the data 
assigned to them? 

That’s correct. Python variables contain a reference to a 
data object.The data object contains the data and, because you 
can conceivably have a string object used in many different places 
throughout your code, it is safest to make ali strings immutable so 
that no nasty side effects occur. 

q lsn’t it a huge pain not being able to adjust strings “in 
place”? 

No, not really. Once you get used to how strings work, it 
becomes less of an issue. In practice, you’ll find that this issue rarely 
trips you up. 


q 


Are any other Python data types immutable? 


Yes, a few. There’s the tuple, which is an immutable list. Also, 
all of the number types are immutable. 

q Other than learning which is which, how will I know when 
something is immutable? 

Don’t worry: you’ll know. If you try to change an immutable 
value, Python raises a TypeError exception. 

q Of course: an exception occurs. They’re everywhere in 
Python, aren’t they? 


A 


I Yes. Exceptions make the world go 'round. 
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persistence 


Kwowmg the type of error is wot ewough 

When a file I/O error occurs, your code displays a generic “File Error” 
message. This is too generic. How do you know what actually happened? 





Who knows? 

It turns out that the Python interpreter knows.. .and it will give up the details 
if only you’d ask. 

When an error occurs at runtime, Python raises an exception of the specific 
type (such as IOError, ValueError, and so on). Additionally, Python 
creates an exception object that is passed as an argument to your except 
suite. 

Let’s use IDLE to see how this works. 


you are here ► 
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idle session 


" / An IDLE Session- 

\ — £ 

Let's see what happens when you try to open a file that doesn't exist, such as a disk file called missing. txt. 
Enter the following code at IDLE's shell: 

»> try: 

data = open('missing.txt') 
print(data.readline(), end='') 
except IOError: 

print('File error') 
finally: 

data.close() 


File error 


TKeves your error messa^e, bui... 



...whai's tWts ? l? A h oihrr M.nl- 

• • • nnotnev excepti oh \was 
Yaaed a«d ii killed your code. 


As the file doesn't exist, the data file object wasn't created, which subsequently makes it impossible to call the 
close () method on it, so you end up with a NameError. A quick fix is to add a small test to the finally 
suite to see if the data name exists before you try to call close () .The locals () BIF returns a collectiori of 
names defined in the current scope. Let's exploit this BIF to only invoke close () when it is safe to do so: 


finally: 

if 



The V operaior iests 
-(•ov 


data' in locals(): 
data.close() 




J 


This is jusi ihe bii oi code thai 
Keeds io chanae. Press A|i_P l a 
edit your Code ai IDLE's shelh 


File error 



No exira excepiions ihis ii me . 
v^ust youir eviror message. 


Here youfe searching the collection returned by the locals () BIF for the string data. If you find it, you can 
assume the file was opened successfully and safely call the close () method. 

If some other error occurs (perhaps something awful happens when your code calls the print () BIF), your 
exception-handling code catches the error, displays your "File error" message and, finally, closes any opened file. 

But you stili are none the wiser as to whot actuolly coused the error. 
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persistence 


When an exception is raised and handled by your except suite, the Python interpreter passes an exception object 
into the suite. A small change makes this exception object available to your code as an identifier: 


except IOError as err: 



print('File error: ' + err) 


<$ve your exdep-fcioh obj edi 
...then use it as pavt 



youv evv-ov- messa^c- 


But when you try to run your code with this change made, another exception is raised: 



Mooys! Vei a*othev > 
extepW; this ti**e 'i s a 
w Ty pcEvv-ov . 


This time your error message didn't appear at ali. It turns out exception objects and strings are not compatible 
types, so trying to concatenate one with the other leads to problems. You can convert (or cast) one to the other 
using the str () BIF: 


except IOError as err: 

print('File error: 



+ str(err)) 


Wse the "strO" BIF io &rde the 
e*deptioh objedi io behave like a sirmg. 


Now, with this final change, your code is behaving exactly as expected: , . /» 

A*d you «o M et a syed, 

message that telis you exadily 

File error: [Errno 2] No such file or directory: 'missing.txt' v/Kat Y/eyvt wVOh<y 



Of course, ali this extra logic is starting to obscure the 
real meaning of your code. 


Wouldnt it be dreamy if there 
were a way to take advantage of 
these mechanisms without the code 
bloat? I guess ifs iust a fantasy... 
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try with 


Use with to work with files 

Because the use of the try/except/finally pattern is so common when 
it comes to working with files, Python includes a statement that abstracts 
away some of the details. The with statement, when used with files, can 
dramatically reduce the amount of code you have to write, because it negates 
the need to include a finally suite to handle the closing of a potentially 
opened data file. Take a look: 


try: 

data = open('its.txt ', "w") 
print("It's... " , file=data) 
except IOError as err: 

print('File error: ' + str(err)) 
finally: 

if 'data' in locals(): 
data.close() 


The use of Vith” 
»e<\a-tes -the need (or 
-the “finally” suite- 


This is the usual "try/ 
c*£ept/-finally" pa tt ev - 



try: 


with open('its.txt' 
print("It's..." 
except IOError as err: 
print('File error: 


"w") as data 
file=data) 

+ str(err) ) 


When you use with, you no longer have to worry about closing any opened 
files, as the Python interpreter automatically takes care of this for you. The 
with code on the the right is identical in function to that on the left. At Head 
First Labs, we know which approach we prefer. 



The with statement takes advantage of a Python technology 
called the context management protocol. 
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persistence 


t^iharpen your pencil 


Grab your pencil and rewrite this try/except/finally code to use 
with instead. Here's your code with the appropriate f inally 
suite added: 



VVvi-bc youv- 
w y/i*bV\ todc 
heve- 
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no finally 



harpen your pencil 

Solutiori 


You were to grab your pencil and rewrite this try/except/finally 
code to use with instead. Here's your code with the appropriate 
finally suite added: 


try: 

man_f ile = open (' man_data. txt ’ , ’ w') 

other_f ile = open (' other_data. txt ’ , ' w') 

print(man, file=man_file) 
print(other, file=other_file) 
except IOError as err: 

print(’File error: ' + str(err)) 
finally: 

if 'man_file' in locals(): 

man_file.close() 
if 'other_file' in locals(): 
other file.close() 


bcf 

UsM VrlW . ,/ p., p., N 

staW^ts {* W'it to*,+il 

^ v/i*th opchOo-thcv- daia-i%i , ) V) as o*thc\r -file: 

“Vmally suite- .. — . — . 

prinifatheir, 'filc =1 o'thcv^_'filc) 
e%£ep*t lOBrror as ev-v*: 

py‘m*trpilc cyyoy: * + s-fo-fm-)) 

Or toJome the two V" 0 ” talis mto °»e /Vote the use o-f the <w a . 

VrlW' staWent 

( f ^ y/i*th opch^mar^data ^t; V) as man_jfile, opch^o-thcy^data t^t; V) as o-they^Pile: 

pvintfman, -file—mar^-Pile) 
pvirrtfoihcv, -file—o-thev^jpile) 
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persistence 



Add your with code to your program, and let’s confirm that it continues to function as expected. Delete the two 
data files you created with the previous version of your program and then load your newest code into IDLE and give 
it a spin. 


N° e^ovs m the I 

sV*\l ayyears -b 


^ ^ ' Python Shell 

Python 3.1.2 (r312:79360M, Mar 24 2010, 01:33:18) 

[GCC 4.0.1 (Apple Inc. build 5493)] on darwin 

Type "Copyright", "credits" or "license()" for more information. 

»> -RESTART- 

»> 

»> 


Ln: 6 Coi: 4 

2 


If you check your folder, your two data files should’ve reappeared. Let’s take a closer look at the data file’s contents 
by opening them in your favorite text editor (or use IDLE). 



['Is this the right room for an argument?', "No you haven't!", 'When?', 
"No you didn't!", "You didn't!", 'You did not!', 'Ah! (taking out his 
wallet and paying) Just the five minutes.', 'You most certainly did not 
!', "Oh no you didn't!", "Oh no you didn't!", "Oh look, this isn't an a 
rgument!", "No it isn't!", "It’s just contradictioni", 'It IS!', 'You j 
ust contradicted me!', 'You DID!', 'You did just then!', '(exasperated) 
Oh, this is futile!!', 'Yes it is!'] 



Heves what the 
said. 


other_data.txt + (-/HeadFirstPython/chapter4) - VIM3 



You Ve saved the lists in two files containing what the Man said and what the Other 
man said. Your code is smart enough to handle any exceptions that Python or 
your operating system might throw at it. 


Well done. This is really coming along. 


you are here ► 


123 


























unsuitable format 


Pefault forwats are unsuitable for files 

Although your data is now stored in a file, it 5 s not really in a useful format. 

Let 5 s experiment in the IDLE shell to see what impact this can have. 

r ' / An IDLE Session- 

\_ f 

Use a with statement to open your data file and display a single line from it: 

»> with open (' man_data. txt') as mdf: . 

print (mdf. readline ()) -- , ° ' 110 hee< * ^ ose y°Wr -file, 

A-^ be^use wi-fch does iuJi U yo , 

['Is this the right room for an argument?', "No you haven't!", 'When?', "No you didn't!", "You 
didn't!", 'You did not!', 'Ah! (taking out his wallet and paying) Just the five minutes.', 

'You most certainly did not!', "Oh no you didn't!", "Oh no you didn't!", "Oh look, this isn't 
an argument!", "No it isn't!", "It's just contradiction!", 'It IS!', 'You just contradicted 
me!', 'You DID!', 'You did just then!', '(exasperated) Oh, this is futile!!', 'Yes it is!'] 


Yikes! It would appear your list is converted to a large string by print () 
when it is saved. Your experimental code reads a single line of data from the 
file and gets ali of the data as one large chunk of text.. .so much for your code 
saving your list data. 

What are your options for dealing with this problem? 



By default, print () displays your data in a format that mimics 
how your list data is actually stored by the Python interpreter. 

The resulting output is not really meant to be processed further... 
its primary purpose is to show you, the Python programmer, 
what your list data "looks like" in memory. 
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I guess I could wri+e some custom 
parsing code to process the "internal 
format" used by "prin+O". It shouldn't 
take me all that long... 


O 



It might be worth looking at 
using something other than a plain 
u print()" to format the data prior 
to saving it to the data file? I'd 
certainly look into it. 


O 



Parsing the data in the file is a possibility.. .although it 5 s complicated by all those 
square brackets, quotes, and commas. Writing the required code is doable, 
but it is a lot of code just to read back in your saved data. 

Of course, if the data is in a more easily parseable format , the task would likely be 
easier, so maybe the second option is worth considering, too? 





Can you think of a function you created from earlier 
in this book that might help here? 
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nonstandard output 


Why not wodify printjold? 

Recall your print_lol () function from Ghapter 2, which takes any list (or 
list of lists) and displays it on screen, one line at a time. And nested lists can 
be indented, if necessary. 

This functionality sounds perfect! Here 5 s your code from the ne ster . py 
module (last seen at the end of Ghapter 2): 


^ " nester.py - /Users/barryp/Downloads/nester-O.O/nester.py 

1111 11 This is the 11 nes ter .py 1,1 module and it provides one function called print lol ( ) 
which prints lists that may or may not include nested lists. 1 ' 1111 

def print lol (the list, indent=False, level=Q): 

11 ““ Prints a list of (possibly) nested lists, 

This function takes a positional argument called "the_list", which 
is any Python list (of - possibly - nested lists). Each data item in the 
provided list is ( recursively ) printed to the screen on it's own line, 

A second argument called "indent" 1 Controls whether or not indentation is 
shown on the display. This defaults to False: set it to True to switch on. 

A third argument called "level" (which defaults to 0) is used to insert 
tab-stops when a nested list is encountered. """ 

for each_item in the list: ^ 

if isinstance( each_item, list): \ 

print lol (each item, indent, level+1) J ,, ,. u v „ 

elseT - / TW.S tode A*r*wUy disylays yoM- 

if indent : > «- jU- or , sdtee«• 

for tab stop in range( level) ; \ 

print ("Xt" , end= rF ) ) 

print (each item) 


Ln: 24 Coi: 0 

A 


Amending this code to print to a disk file instead of the screen (known as 
Standard output) should be relatively straightforward. You can then save your 
data in a more usable format. 




Standard 0u*tpu*t I he de-Pault plade where your todt wv-ftes i-ts 
data wher\ the w prm*tO w BIF is used- This is -typidally the sev-ee*. 
I* Pytho*, sta*dard output is reterred to as w sys.stdout w a*d 
is impov-table -from the Standard Library *s w sys w module- 
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Let’s add a fourth argument to your print_lol () function to identify a place to write your 
data to. Be sure to give your argument a default value of sys. stdout, so that it continues to 
write to the screen if no file object is specified when the function is invoked. 


Fili in the blanks with the details of your new argument. (Note: to save on space, the comments 
have been removed from this cod, but be sure to update your comments in your nes ter. py 
module after you’ve amended your code.) 


def print lol (the list, indent=False, level=0. 


for each_item in the_list: 

if isinstance(each_item, list): 

print_lol(each_item, indent, level+1 
else: 

if indent: 

for tab_stop in range(level): 
print("\t", end='', 
print (each item. 


) 


) 


) : 


What needs to happen to the code in your with statement now that your amended print_lol () 
function is available to you? 


List the name of the module(s) that you now need to import into your program in order to support your 
amendments to print lol(). 


you are here ► 
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extend your function 



ExwctSe 


§ 0 (.|#tl 0 H 


You were to add a fourth argument to your print_lol () function to identify a place to write 
your data to, being sure to give your argument a default value of sys . stdout so that it 
continues to write to the screen if no file object is specified when the function is invoked. 


You were to fili in the blanks with the details of your new argument. (Note: to save on space, the 
comments have been removed from this code, but be sure to update those in your nester. py 
module after you’ve amended your code). 

Add the Wth av^wmewt a »d <yve 't a 


def print lol(the list, indent=False, 


de-Caulb value 

level=0, 


A 

lh—sys.s-tdou-t 


for each_item in the_list: 

if isinstance(each_item, list): 

print_lol(each_item, indent, level+1 
else: 





Noic- -the 
signature has 

dhanged- 


if indent: 

for tab_stop in range(level): 

print ("\t", end='', -f ile—-CVi 

print(each item, 'file= 1 'fh 


<r 


Ad\ust -the -two 
dalls to w pr’»ntO 
to use *the new 


av^umen 


t- 


What needs to happen to the code in your with statement now that your amended print_lol () 
function is available to you? 


The dode needs t o be adjusted so that instead o-f using -the 
w printO w 3IF) the dode needs to invoke u p\rin*t_lolO^ instead- 


Q List the name of the module(s) that you now need to import into your program in order to support your 
amendments to print lol(). 


The program needs to imj>o\rt the amended M nestev w module- 
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Before taking your code for a test drive, you need to do the following: 

1. Make the necessary changes to nester and install the amended module into your Python 
environment (see Chapter 2 for a refresher on this). You might want to upload to PyPI, too. 

2. Amend your program so that it imports nester and uses print_lol () instead of print () 
within your with statement. Note: your print_lol () invocation should look something like this: 

print_lol(man, fh=man_file). 

When you are ready, take your latest program for a test drive and let’s see what happens: 


be-Cov-e, ‘tbeve s no 
ou-b[>ut on sdvccn- -- 


Ln: 6 Coi: 4 


^ Python Shell 

Python 3.1.2 (r312:79360M, Mar 24 2010, 01:33:18) 

[GCC 4.0.1 (Apple Inc. build 5493)] on darwin 

Type "Copyright", "credits" or "license()" for more information. 
»> ================================ RESTART =================== 

»> 

»> 


Lefs check the contents of the files to see what they look like now. 


^ ^ ^ man_data.txt + (~/HeadFirstPython/chapter4-original) - VIM1 

t"i Hri ^ — — * X 


Is this the right room for an argument? 

No you haven't! 

When? 

No you didn’t! 

You didn't! 

You did not! 

Ah! (taking out his wallet and paying) Just the five minutes. 
You most certainly did not! 

Oh no you didn't! 

Oh no you didn't! 

Oh look, this isn't an argument! 

No it isn't! 

It's just contradictioni 
It IS! 

You just contradicted me! 

You DID! 

You did just then! 

(exasperated) Oh, this is futile!! 

Yes it is! 



\ 


What -the r*an 

said is now 

Icgiblc. 




^ ' other_data.txt + (~/HeadFir...thon/chapter4-original) - VIM2 

* id Hei %) w ©• L_=j iL=j t X 


I’ve told you once. 

Yes I have. 

Just now. 

Yes I did! 

I’m telling you, I did! 

Oh I’m sorry, is this a five minute argument, or the full half hour? 
Just the five minutes. Thank you, 

Anyway, I did. 

Oh yes I did! 

Oh yes I did! 

Yes it is! 

No it isn‘t! 

It is NOT! 

N° I didn t! the other 

No no no! ^ 

Nonsense! 

No it isn’t! 

I 


And heirVs what 


man 



said- 


▲ 

T 


This is looking good. By amending your nester module, you Ve provided a 
facility to save your list data in a legible format. It 5 s now way easier on the eye. 


But does this make it any easier to read the data back in? 


you are here ► 
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brittle code 



Hang on a second...haven'+ you been 
here before? YouVe already written 
code to read in lines from a data file and 
put 'em in+o lis+s...do you like going around 
in circles?!? 


That’s a good point. 

This problem is not unlike the problem from the 
beginning of the chapter, in that you Ve got lines of 
text in a disk file that you need to process, only now 
you have two files instead of one. 

You know how to write the code to process your 
new files, but writing custom code like this is 
specific to the format that you Ve created for this 
problem. This is brittle : if the data format changes, 
your custom code will have to change, too. 

Ask yourself: is it worth it? 
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Custom Coie Exposed 

This week’s interview: 

When is custom code appropriate? 


Head First: Helio, GG, how are you today? 

Custom Code: Hi, I’m great! And when I’m not 
great, there 5 s always something I can do to fix things. 
Nothing 5 s too much trouble for me. Here: have a 
seat. 

Head First: Why, thanks. 

Custom Code: Let me get that for you. It 5 s my 
new custom SlideBack&Groove™, the 2011 model, 
with added cushions and lumbar support.. .and it 
automatically adjusts to your body shape, too. How 
does that feel? 

Head First: Actually [relaxes], that feels kinda 
groovy. 

Custom Code: See? Nothing 5 s too much trouble 
for me. I’m your “go-to guy.” Just ask; absolutely 
anything 5 s possible when it 5 s a custom job. 

Head First: Which brings me to why I’m here. I 
have a “delicate” question to ask you. 

Custom Code: Go ahead, shoot. I can take it. 

Head First: When is custom code appropriate? 

Custom Code: Isn’t it obvious? IEs always 
appropriate. 

Head First: Even when it leads to problems down 
the road? 

Custom Code: Problems?!? But Fve already told 
you: nothing 5 s too much trouble for me. I live to 
customize. If it 5 s broken, I fix it. 

Head First: Even when a readymade solution 
might be a better fit? 

Custom Code: Readymade? You mean (I hate to 
say it): off the shelfi 

Head First: Yes. Especially when it comes to 
writing complex programs, right? 


Custom Code: What?!? ThaEs where I excel: 
creating beautifully crafted custom Solutions for folks 
with complex computing problems. 

Head First: But if something 5 s been done before, 
why reinvent the wheel? 

Custom Code: But everything I do is custom- 
made; thafs why people come to me... 

Head First: Yes, but if you take advantage of other 
coders’ work, you can build your own stuff in half 
the time with less code. You can’t beat that, can you? 

Custom Code: “Take advantage”...isn’t that like 

exploitation ? 

Head First: More like collaboration, sharing, 
participation, and working together. 

Custom Code: [shocked] You want me to give my 
code... away ? 

Head First: Well.. .more like share and share alike. 
Eli scratch your back if you scratch mine. How does 
that sound? 

Custom Code: That sounds disgusting. 

Head First: Very droll [laughs]. All Em saying is 
that it is not always a good idea to create everything 
from scratch with custom code when a good enough 
solution to the problem might already exist. 

Custom Code: I guess so.. .although it won’t be as 
perfect a fit as that chair. 

Head First: But I will be able to sit on it! 

Custom Code: [laughs] You should talk to my 
buddy Pickle. . .he 5 s forever going on about stuff like 
this. And to make matters worse, he lives in a library. 

Head First: I think Eli give him a shout. Thanks! 

Custom Code: Just remember: you know where to 
find me if you need any custom work done. 


you are here ► 
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in a pickle 


Pickle your data 


Python ships with a Standard library called pickle, which can save and load 
almost any Python data object, including lists. 

Once you pickle your data to a file, it is persistent and ready to be read into 
another program at some later date/time: 


Your data as it appears 
'm Python s roeroory 


The piekle engine 


['Is this the right room for an 
argument?', "No you haven't! " , 
'When?', "No you didn't!", "You 
didn't!", 'You did not!', 'Ah! 
(taking out his wallet and paying) 
Just the five minutes.', 'You 
most certainly did not!', "Oh 
no you didn't!", "Oh no you 
didn't!", "Oh look, this isn't 
an argument!", "No it isn't!", 
"It's just contradiction!", 'It 


IS! ' , 
me! ' , 
then! 


'You just contradicted 
'You DID!', 'You did just 
, '(exasperated) Oh, this 


is futile!! 


Yes it is!'] 



Feed your Pythoh 
data to pickle. 


Out comes the 
pickled version ot 



You can, for example, store your pickled data on disk, put it in a database, 
or transfer it over a network to another computer. 


When you are ready, reversing this process unpickles your persistent pickled 
data and recreates your data in its originalform within Python’s memory: 




The same pickle engihe 


—►- 


Feed your pickled 
data to pickle. 


0ut comes the 
Python version o 
your pitkled data 


Your data is recreated 
in Python s memory, 
exactly as be-fore. 


['Is this the right room for an 
argument?', "No you haven't!", 
'When?', "No you didn't!", "You 
didn't!", 'You did not!', 'Ah! 

(taking out his wallet and paying) 
Just the five minutes.', 'You 
most certainly did not!', "Oh 
no you didn't!", "Oh no you 
didn't!", "Oh look, this isn't 
an argument!", "No it isn't!", 
"It's just contradiction!", 'It 
IS!', 'You just contradicted 
me!', 'You DID!', 'You did just 
then!', '(exasperated) Oh, this 

is futile!!', 'Yes it is!'] 
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Save with dump and restore with load 


Using pickle is straightforward: import the required module, then use 
dump () to save your data and, some time later, load () to restore it. The 
only requirement when working with pickled hies is that they have to be 
opened in binary access mode : 


Always 

“to import 
the "pickle" 
module. 

To save your data, 
use U dump0 . 


import pickle 

r • • • 

with open('mydata.pickle ', 'wb') as mysavedata: 

pickle.dump([1 , 2 , ’three '], mysavedata) 

• • • _ 
with open (’ mydata. pickle ' , 1 ' rb') as myrestoredata: 
a_list = pickle.load(myrestoredata) 




f «w youv daia tvom youv . 
•file using "loadO" ' 


Out youv daia i* bah m youv pvogva*, you 
iveai ii like a«y othev daia objedt- 


The V telis 
Python to open 
youv daia tiles 

i» bimary 

mode 


What if sowething goes wrong? 


If something goes wrong when pickling or unpickling your data, the pickle 
module raises an exception of type PickleError. 



Here's a snippet of your code as it currently stands. Grab your 
pencil and strike out the code you no longer need, and then 
replace it with code that uses the facilities of pickle instead. 
Add any additional code that you think you might need, too. 


try: 

with open('man data.txt', 'w') as man file, open('other data.txt', 'w') as other file: 
nester.print lol(man, fh=man file) 
nester.print lol(other, fh=other file) 
except IOError as err: 

print('File error: ' + str(err)) 


you are here ► 
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significance of the pickle 


Sharpen your pencil 

Solutiori 



vT 

pidkle 


/***po*-t "pidkle" 
o( you*- 


Ke<3v 


Here's a snippet of your code as it currently stands. You were to grab 
your pencil and strike out the code you no longer need, and then 
replace it with code that uses the facilities pickle instead. You 
were also to add any additional code that you think you might need. 


try: \wb' '^b' 

with open('man data.txt', ) as man_file, open('other_data.txt ■' T ) as other_file 

pidkle-dumf(main, m<3*i_-file) 



Cbawy ibe attess mode -to 
be "vjvi-teable, bmary • 



„iiuu Lli . [ jii iliL IUI (man, ili-iu' 
■ acator.print lol(othor; — fh otlrc? 

except IOError as err: 

print('File error: ' + str(err)) 

pidklc PidklcEv-V-o\r as pc\rv-: 
pvirrt^Pidkling c\r\ro\r-* 1 + s*t\rf 


) pidklc dum 


Reylate -tbe buo talis -to nestevfv-m-tJoK) 
y/tb talis b> l y.tkle dumyO . 


h bandle a»y e *te P tio»s 
that odduir. 


thereicffe no 

Dumb QuestiQtis 


q When you invoked print_lol() earlier, you provided only two arguments, even though the function signature requires you to 
provide four. How is this possible? 


When you invoke a Python function in your code, you have options, especially when the function provides default values for some 
arguments. If you use positional arguments, the position of the argument in your function invocation dictates what data is assigned to which 
argument. When the function has arguments that also provide default values, you do not need to always worry about positional arguments 
being assigned values. 



OK, you’ve completely lost me. Can you explain? 


Consider print (), which has this signature: print (value, sep=’ ', end=' \n', f ile=sys . stdout ). By 

default, this BIF displays to Standard output (the screen), because it has an argument called file with a default value of sys . stdout. 
The file argument is the fourth positional argument. However, when you want to send data to something other than the screen, you do not need 
to (nor want to have to) include values for the second and third positional arguments. They have default values anyway, so you need to provide 
values for them only if the defaults are not what you want. If all you want to do is to send data to a file, you invoke the print () BIF like this: 
print("Dead Parrot Sketch", file= ’ myfavmonty. txt ’) and the fourth positional argument uses the value 
you specify, while the other positional arguments use their defaults. In Python, not only do the BIFs work this way, but your custom functions 
support this mechamism, too. 
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Let’s see what happens now that your code has been amended to use the Standard pickle module instead of 
your custom nester module. Load your amended code into IDLE and press F5 to run it. 


Onu a$a'm, you 
no visual ^ 
i\\di somC-tKrnJ ^ 


^ O O Python Shell 

Python 3.1.2 (r312s79360M, Mar 24 2010, 01:33:18) 

[GCC 4.0.1 (Apple Inc. build 5493)] on darwin 

Type "Copyright", "credits" or "license()" for more information. 

»> -RESTART- 

»> 

»> 



Ln: 6 Coi: 4 

4 


So, once again, let’s check the contents of the files to see what they look like now: 


The is -the ma*/ 

pi^klcd da-fca. 


« n n man_data.txt + (~/HeadFirstPython/chapter4-original) - VIM1 

.. w w - y» w ~ e m m * * i ^ 

<80^'"C ]q" 3 (X’ FjF(F(3ls this the right room for an argument?q~AX A 0 A (a A 3 A (aNo ^ 

you haven't!q'BX^E^(aMWhen?q A CX A N^(a A (aNo you didn't!q A DX A K A (r(T<aYou di 
dn't !q A EX A L A @ A <a A @You did not !q A FX= A (a A <sr@Ah! (taking out his wallet and pa 
ying) Just the five minutes.q A GX A [ A @ A <a A (aYou most certainly did not!q~HX A Q 
A <a A (a A (aOh no you didn't!q X A Q A (a A <a A (aOh no you didn't!q 
X A (a A (a A @0h look, this isn't an argument !q A KX A L A <a A <a A @No it isn't !q A LX A X A (a' 

@ A <ait's just contradiction! q A MX A F A <a A <a A <alt — y - y-~ ;- . , . - . 1V , 

icted me!q A OX A H A ta A <T<aYou DID! q A PX A R A <a A <a^You c m other_data.txt + (~/HeadFirstPython/chapter4-original) - VIM2 

asperated) Oh, this is futile!!q A RX I J 1« 

A <a A @ A (aYes it is!q A Se. i*l H H %f imi wtm a 



The is the othev 
man's ^itkled data- 


<80> A C]q A (a(X A S A (a A <jr(al ' ve told you once. q A AX A K A <a A (a A <aYes I have.q A BX A (a A @ A 
(aJust now.q A CX 

A <a A @ A (aYes I did! q A DX A W A @ A @ A (al 'm telling you, I did! q A EXD A (a A (a A (aOh l'm sorr 
y, is this a five minute argument, or the full half hour?q A FX! A <a A <a A @Just 
the five minutes. Thank you.q A GX A N A @ A (a A (aAnyway, I did.q A HX@ A (a A (a A (aNow let' 
s get one thing quite ciear: I most definitely told you!q X A M A (a A (a A @0h 
yes I did!q 

X A M A <a A <T@0h yes I did!q A KX 

A (a A (a A (aYes it is !q A LX A L A (a A <jT@No it isn't!q A NX 

A <a A @ A (alt is NOT!q A NX A L A <a A (a A @No I didn't!q A 0X A (a A (jT@No no no!q A PX A <a A <a A 
(aNonsense!q A QX A L A (a A @ A (aNo it isn't!q A Re. 


A 


It appears to have worked.. .but these files look like gobbledygookl What gives? 

Recall that Python, not you, is pickling your data. To do so efficiently, Python’s 
pickle module uses a custom binary format (known as its protocol). As you 
can see, viewing this format in your editor looks decidedly weird. 


Don’t worry: it is supposed to look like this. 


you are here ► 
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idle session 


An (DLE Session 

r 


pickle reaIly shines when you load some previously pickled data into another program. And, of course, there's 
nothing to stop you from using pickle with nester. After ali, each module is designed to serve different 
purposes. Let's demonstrate with a handful of lines of code within IDLE's shell. Start by importing any required 
modules: 

»> import pickle 
»> import nester 

No surprises there, eh? 

Next up: create a new identifier to hold the data that you plan to unpickle.Create an empty list called new_man: 
»> new_man = [] 

Yes, almost too exciting for words, isn't it? With your list created. Iet's load your pickled data into it. As you are 
working with external data files, it's best if you enclose your code with try/except: 

»> try: 

with open('man_data.txt', 'rb') as man_file: 

new_man = pickle.load(man_file) 
except IOError as err: 

print('File error: ' + str(err)) 
except pickle.PickleError as perr: 

print('Pickling error: ' + str(perr)) 

This code is not news to you either. However, at this point, your data has been unpickled and assigned to the 
new_man list. It's time for nester to do its stuff: 

»> nester.print_lol (new_man) 

Is this the right room for an argument? 

No you haven't! 

When? 

No you didn't! 


Noi all -the da-fca is shovm 

hev-e, bu-fc -forus-fc us- i-t's ali 
■fcheire. 


You did just then! 

(exasperated) Oh, this is futile!! 

Yes it is! 

And to finish off, let's display the first line spoken as well as the last: 

»> print (new_man [ 0 ]) 

Is this the right room for an argument? 

»> print (new_man [ -1 ]) 

Yes it is! 
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fteweric file 1/0 with pickle is the way to go! 


Now, no matter what data you create 
and process in your Python programs, 
you have a simple, tested, tried- 
and-true mechanism for saving and 
restoring your data. How cool is that? 


O 



Python takes care of your file I/O details, so you can concentrate on what 
your code actually does or needs to do. 

As you Ve seen, being able to work with, save, and restore data in lists is a 
breeze, thanks to Python. But what other data structures does Python 
support out of the box ? 

Let’s dive into Chapter 5 to find out. 


you are here ► 
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Your Pythow Toolbox 

You’ve got Chapter 4 under your 
belt and you’ve added some key 
Python techiques to your toolbox. 


«i_ u taV>\< 




r ( . data 

ov>tc ass^rved 

«* i i ljvc vaWe 

a vaWc, «"" ot 

, , c? votesso^ 

# " \ \ Jtp ferite** 

• a data o®d et 

sav/»^ a »«• 

stpvay- ot 

„ IA.* oy-OCC^ v 

•rtr^ *** ^ 

Jfvo* 


<^\ BUL1ET 


POINIS 


■ The strip () method removes 
unwanted whitespace from strings. 


■ The file argument to the print () 
BIF Controls where data is sent/saved. 


■ The f inally suite is always executed 
no matter what exceptions occur within a 
try/except statement. 

■ An exception object is passed into the 
except suite and can be assigned to 
an identifier using the as keyword. 

■ The str () BIF can be used to access 
the stringed representation of any data 
object that supports the conversion. 

■ The locals () BIF returns a collection 
of variables within the current scope. 

■ The in operator tests for membership. 

■ The “+” operator concatenates two 
strings when used with strings but adds 
two numbers together when used with 
numbers. 


■ The with statement automatically 
arranges to close ali opened files, even 
when exceptions occur. The with 
statement uses the as keyword, too. 

■ sys . stdout is what Python calls 
“Standard output” and is available from 
the Standard library’s sys module. 

■ The Standard library’s pickle module 
lets you easily and efficiently save and 
restore Python data objects to disk. 

■ The pickle. dump () function saves 
data to disk. 

■ The pickle . load () function 
restores data from disk. 
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5 coiT]pre}ienc[Ing data 


^ Work that data! + 



Data comes in all shapes and sizes, formats and encodings. 

To work effectively with your data, you often have to manipulate and transform it into a 
common format to allow for efficient Processing, sorting, and storage. In this chapter, you’ll 
explore Python goodies that help you work your data up into a sweat, allowing you to 
achieve data-munging greatness. So, flip the page, and let’s not keep the coach waiting... 


this is a new chapter 
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coaching crisis 


Coach Kelly weeds your help 



I'm too busy on the track 
to waste time f iddling with 
my computer. Can you help 
me process my athlete data? 


The coach is an old friend, and you’d love to help. His crack squad of U10 
athletes has been training hard. With each 600m run they do, Coach Kelly 
has recorded their time in a text file on his computer. There are four files in 
all, one each for James, Sarah, Julie, and Mikey. 



jdmes.-tx-fc 


Initially, the coach needs a quick way to know the top three fastest times for each 
athlete. 

Can you help? 
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comprehending data 



Before proceeding with this 
chapter, take a few moments 
to download the four data 
files from the Head First 
Python support website. 



Let’s begin by reading the data from each of the files into its own list. Write a short program to 
process each file, creating a list for each athlete’s data, and display the lists on screen. 

Hint: Try splitting the data on the commas, and don’t forget to strip any unwanted whitespace. 


VVvrte youv 
dode heve. 


you are here ► 
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><o 


let’s split 



SotutioH 


0pen eafch o-f 
-the data -f iles 

m tiAVn, vcad 
-the Ime o-f 
data -fvom the 
-f ile, a^d fcv-eate 
a list -fv-om the 
|*me o$ data- 


Let’s begin by reading the data from each of the files into its own list. You were to write a short 
program to process each file, creating a list for each athlete’s data. You were then to display the 
lists on screen. ^-- 0p Ch ^e -Pile. 



pv-irrt(james) 
printfjulie) 

j>v'm-fc(savah) 


tliere jare n° 

Dtimb Questi9ns 


That data. strip () . split (’, 1 ) line looks a little weird. Can you explain what’s going on? 

That’s called method chaining. The first method, strip (), is applied to the line in data, which removes any unwanted whitespace 
from the string. Then, the results of the stripping are processed by the second method, split (', 1 ), creating a list. The resulting list is 
then applied to the target identifier in the previous code. In this way, the methods are chained together to produce the required resuit. It helps 
if you read method chains from left to right. 
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comprehending data 



Load your code into IDLE and run it to confirm that it's all OK for now: 


coach.py - /Users/barryp/HeadFirstPython/chapterS/coach.py 

with open( 1 james.txt 1 ) as jaf: 

data = jaf.readline() 
james = data.strip().split( 1 , 1 ) 

with open ( 1 julie.txt 1 ) as juf: 

data = juf.readline(} 
julie = data. strip( ) . spl.it ( 1 , 1 ) 

with open ( 1 mikey.txt 1 ) as mif: 

data - mif.readline() 
mikey = data.strip(}.split( ' , 1 } 

with open ( 1 sarah.txt 1 ) as saf: 

data = saf.readline() 
sarah = data.stripf).split(', 1 ) 

print (james) 
print (julie) 
print (mikey) 
print (sarah) 

j_ 

^ ^ _ Python Shell _ 

Python 3.1.2 (r3l2:79360M, Mar 24 2010, 01:33:16) 

[GCC 4.0.1 (Apple Inc. build 5493)] on darwin 

Type "Copyright " , "credits" or "license( ) " for more information, 

>» =»»»»«»«»»»»»»»«= restart «=— 

»> 


[ '2-34 ’ , 

'3:21 F , 

'2*34 F , 

'2*45 F , 

'3*01 F , 

'2:01 F , 

'2i0l F , 

'3il0 F , 

'2-22 ' ] 

[ '2.59 ’ , 

'2*11' , 

' 2:11 F , 

'2£23 F , 

'3-10 F , 

' 2-23 ' , 

'3:10 ' , 

'3*21 F , 

'3-21 ' ] 

[ '2:22 ’ , 

'3*01 F , 

'3i01 F , 

'3.02 F , 

' 3 £ 02 ' , 

'3*02 F , 

' 3 £ 2 2 ’ , 

'2*49 F , 

r 2 £ 3 8 ' ] 

[ '2:58 ' , 

F 2*58 F , 

' 2 i 3 9 F , 

'2-25 F , 

'2-55 ' , 

' 2:54 ' , 

' 2*18 ' , 

'2£55 ' , 

'2£55 1 ] 


»> 


Ln: 10 Coi: 4 


your 
prograr» a s 

displaycd ih 
IDLE. v 



A»d heres ihe ou-t^wt 
^rodwted loy ruw»w$ 



So far, so good. Goach Kelly 5 s data is now represented by four lists in Python 5 s 
memory. Other than the use of method chaining, there 5 s nothing much new 
here, because youVe pretty much mastered reading data from files and using 
it to populate lists. 

There’s nothing to show the coach yet, so no point in disturbing him until his 
data is arranged in ascending order, which requires you to sort it. 

Let’s look at your sorting options in Python. 
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in-place or copied sorting 


Sort iw owe of two ways 

When it comes to sorting your data using Python, you have two options. 

In-place sorting takes your data, arranges it in the order you specify, and 
then replaces your original data with the sorted version. The original ordering 
is lost. With lists, the sort () method provides in-place sorting: 


The 

^ovdeved data 



l 


\ < 



4. 






J} e Pyiho» '7 h _pU. c 

r 


es. 


The original daia has 

how been ovdeved (and 
veplaced). 


Copied sorting takes your data, arranges it in the order you specify, and 
then returns a sorted copy of your original data. Your original data’s ordering 
is maintained and only the copy is sorted. In Python, the sorted () BIF 
supports copied sorting. 


The ov^mal, 
pnordeved daia 



l 




4. 






' h9me ^ ^ 


The original, unovdeved daia 

ren>aihs WTOUCHBD- 


The daia has ^ 

ovdeved (a*d top>ed 
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comprehending data 


/ An (DLE Session 

P 


Let's see what happens to your data when each of Python's sorting options is used. Start by creating an unordered 
list at the IDLE shell: 

»> data = [6, 3, 1, 2, 4, 5] Cvedte £ list o-p 

»> data r uhotdeted data and 

[6, 3, i, 2, 4, 5] ) assijh -to a vatiable- 

Perform an in-place sort using the sort () method that is built in as Standard to every Python list: 

’ Vfo ~ sort», o» 

The datas ov-dev-mg has ehanged. 


»> data. sort () 
»> data 


[1, 2 , 3 , 4, 5, 6] 

Reset data to its original unordered state, and then perform a copied sort using the sorted () BIF: 


»> data = [6, 3, 1, 2, 4, 5] 
»> data 

[6, 3, 1, 2, 4, 5] 

»> data2 = sorted (data) 


V 

V 

V 

data 




[6, 

3 , 1, 

2, 

4 , 

5] 

V 

V 

V 

data2 




[1, 

2 3 

r r 

4, 

5, 

6] 




The datas otdetmO) has been reset. 
Pet-fotw, COPIBD sorting o* the data. 

Same as it evev" was. 

The fcopied data is ordered 
tv-orn lowest to highest- 



Either sorting option works with the coach's data, but let's use a 
copied sort for now to arrange to sort the data on output. In the 
space below, provide four amended print () statements to 
replace those at the bottom of your program. 
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><o 


all sorted 



Either sorting option works with the coach's data, but let's use 
a copied sort for now to arrange to sort the data on output. You 
were to provide four amended print () statements to replace 
those at the bottom of your program. 


sw ? i Y «ii . C ^ mi(s ^ 0 arnes)) 

SO»"fced0 t he J 

data WrOVZ- ^rrr-rr. - /..J. ... J . 

you dis^laY 0YX y p\r'm*t^so\rtcd^ikcy)) 

sCree*. ./.’ TI ‘‘". 

p\r'm*t(so\rtcd(sav-ak)) 


tWei are no 

Dumb Questipns 


What happens to the unsorted data when I use sort()? 

For all intents and purposes, it disappears. Python takes a copy, 
sorts it, and then replaces your original data with the sorted version. 



And there’s no way to get the original data back? 


No. If the ordering of the original data is important to you, use 
the sorted () BIF to transform your data into a sorted copy. 



YouVe already seen method chaining, and now it's time to say 
"hello"to function chaining. Function chaining allows you to apply 
a series of functions to your data. Each function takes your data, 
performs some operation on it, and then passes the transformed 
data on to the next function. Unlike method chains, which read 
from left to right, function chains read from right to left (just to 
keep things interesting). 
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comprehending data 



Let's see if this improves your output in any way. Make the necessary amendments to your code and 
run it. 


coach2.py - /Users/barryp/HeadFirstPyth 


i / i.- ii ayj lci j / i c. , 


with open( p james*txt' } as ja£: 

data = jaf.readline() 
james = data.strip().split( 1 f ' 

with open( 1 julie*txt p } as ju£: 

data = juf.readlinef) 
julie = data.strip().split( 1 # ' 


) 


with open( 1 mikey.txt 1 ) as mi£: 

data = mif.readlinef) 
mikey = data.stripf).split( 1 , 1 ) 

with open( 1 sarah.txt 1 ) as saf: 

data = saf.readlinef) 
sarah = data.strip().split( 1 f 1 ) 


print ( sorted (j ames)) 
print ( sorted (j alie)) 
print ( sorted (mikey)) 
print ( sorted (sarah)) 


\\evc s the 
\p the to&* 


date* 



look ai tH/S/ 

1 he ( M i ai 

So * r ted...whic(, j s , | ik( 
wci rd. 


J 


«oo 


Python Shell 


>» == 
»> 

[ 1 2-22 
[ p 2 - 2 3 
[ p 2 * 4 9 
[ 1 2-25 
»> I 


RESTART 


2-34 
2*11 
2 12 2 
2-55 


Look at tW.s: T-& is 

60 ^ UQ..."*» 

tV>at is v»ei*-d- 



2.45 
2s 11 
3*01 
2*58 


2 i 01 p f 

2:23 p , 
3*02 p , 
2:39 \ 


2 ; 01 
3-10 
3.02 
2; 54 


p 3*01 p , 
'3-21 ' , 
p 3:01 p , 
p 2:55 F , 


3 i 10 p , 
3*21 p f 
3 i 0 2 1 f 
2: 55 p , 


3 j 21 p ] 
3:IO 1 ] 
3 12 2 p ] 
2:58 p ] 


Ln: 10 Coi: 4 


A 



Hey, it looks like your data values are 
not uniform. Is the problem with all those 
periods, dashes, and colons? 


Yes. The minute and seconds separators are 
confusing Python’s sorting technology. 

When recording his athletes’ times in each of their files, Goach 
Kelly sometimes used a different character to separate minutes 
from seconds. It looks like you need to fix your data. 
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time trials 


The trouble with time 


Well.. .there 5 s never enough of it, is there? 


Lef s look closely at the coaclfs data to see what the problem is. Here 5 s Sarah 
raw data again: 



saralvtx-fc 



0 


Recall that data read from a file comes into your program as text , so Saralfs 
data looks like this once you turn it into a list of “times”: 


['2:58', '2.58', '2:39', '2-25', '2-55', '2:54', '2.18', '2:55', '2:55'] 




f 

These are all sferiny, eve* toatM 

■tWmks tVieyVe Wes. 



And when you sort Saralfs data, it ends up in this order (which isn’t quite 
what you were expecting): 


['2-25', '2-55', '2.18', '2.58', '2:39', '2:54', '2:55', '2:55', '2:58 


( A 

Woops/ Uai's 
irigU- fiow 2./« 
tome a-f-tev- 2-55? 


^ IA/Koo^s iw* SSS\ 

ta*'t 6<we beWe« 
2.5« a«d «*> vt? 



Python sorts the strings, and when it comes to strings, a dash comes before a 
period, which itself comes before a colon. As all the strings start with 2, the 
next character in each string acts like a grouping mechanism, with the dashed 
times grouped and sorted, then the period times, and finally the colon times. 


Nonuniformity in the coach’s data is causing the sort to fail 
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comprehending data 



Cocte Magnets 


Let's create a function called sanitize (), which takes as input 
a string from each of the athlete's lists. The function then processes 
the string to replace any dashes or colons found with a period and 
returns the sanitized string. Note: if the string already contains a 
period, there's no need to sanitize it. 


Rearrange the code magnets at the bottom of the page to provide 
the required functionality. 


def sanitize(time string): 


return(mins + '+ secs) 


ReU» -the sahifczcd W sbrm b> 

«llw ^ -th.s Uubo». 9 


y/artmj- 



ave 
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sanitzing function 



Code Magnets Solution 

You were to create a function called sanitize (), which takes 
as input a string from each of the athlete's Iists. The function then 
processes the string to replace any dashes or colons found with a 
period and returns the sanitized string. Note: if the string already 
contains a period, there's no need to sanitize it. 


You were to rearrange the code magnets at the bottom of the 
previous page to provide the required functionality. 


def sanitize(time string): 


Use i\\t V' 
opev-a-tov- “to 
cMtcV i-f -the 
sbrm^ dowtaifiS 
a dash ov a 
6oloy\. 



elif ' : ' i n time__string: 
splitter = ':' | 



^■0 turn (tino_string) 


Do hotiiihj i-P the string does 

NOT need to be sanitized. 


(mins , secs) 


time string.split(splitter) 


return(mins + '+ secs) 


Sflit 




‘the stvm^ "t® e^t^att “the 
tes and setonds \>avb- 


Of course, on its own, the sanitize () function is not enough. You need 
to iterate over each of your lists of data and use your new function to convert 
each of the athlete 5 s times into the correct format. 


Let’s put your new function to work right away. 
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comprehending data 



Let’s write the code to convert your existing data into a sanitized version of itself. Create four 
new lists to hold the sanitized data. Iterate over each athlete’s list data and append each 
sanitized string from each list to the appropriate new list. Conclude your program by printing a 
sorted copy of each new list to the screen. 


The to&t that 
v-eads “the data 
-fvow -the data 
-files \rew>ams 
unehanoycd 
(and has been 
dom^v-essed to -f it 
on -this pay). 


with open('james.txt') as jaf: data 
james = data.strip().split) 
with open('julie.txt') as juf: data 
julie = data.strip().split) 
with open('mikey.txt') as mif: data 
mikey = data.strip() .split ) 
with open('sarah.txt') as saf: data 
sarah = data.strip().split) 


j af.readline() 


juf.readline() 


mif.readline() 


saf.readline() 


/\<M *** 

tode V*ere- ^ 


happens 

w "the -four 

VmtO" 

state»weivts? ' 


print( 
print( 


print( 


print( 


) 

) 

) 

) 
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sanitized for your protection 



SoLutiort 


Let’s write the code to convert your existing data into a sanitized version of itself. You were to 
create four new lists to hold the sanitized data. You were then to iterate over each athlete’s data 
and append each sanitized string from each list to an appropriate new list. You were to conclude 
your program by printing a sorted copy of each new list to the screen. 


with open('james.txt') as jaf: data 
james = data.strip().split(',') 
with open ('julie.txt') as juf: data 
julie = data.strip() .split ( ', ') 
with open('mikey.txt') as mif: data 
mikey = data.strip() .split ( ', ') 
with open('sarah.txt') as saf: data 
sarah = data.strip().split ) 


Cveat-e -fouv 
kcw, mrbally 
emfty list*. 




dle3r\__j3mes ^ CJ 
dlc3r\_julic =• □ 
de3n_jr*ikey =■ □ 
dlc3n s3V~3h =■ □ 


j af.readline() 


j uf.readline() 


mif.readline() 


saf.readline() 


nc w 

sta-fcerKCK-fcs how 

dispby -the hCW ^ 
lists; whidh 3 v~c 

soirted, loo. 


£ o\r e3dh__*t in j3mcs: 

dlc3r\_j3mcs.3ppcr\d^ s 3r\i*tiz^^C3dk_jt)) 
-for e 3 dh_jt in julic : 

dlg3n^ulig. 3pfgnd (s3ni-ti2^(g3d^ _t)) 
•rov each_t in rr»ikcy : 

clean_mikeyappend(sanitiz-etach_t) 

•fov- each t in sarah: ^ 




clean sarah.append(saniti«(each -t)) 


print( 

so\rted(de 3 n_j 3 r*es) 

) 

print( 

sov-*ted(dle 3 n_julie) 

) 

print( 

so\rtcd^dlc 3 n_mikcy) 

) 

print( 

sov-*ted(dle 3 n__s 3 v- 3 h) 

) 


Take each of -the data items in 
'the original lists, sanitize the*, 
and.then append the sanitized 
data to the appropriate new 
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comprehending data 



Combine your sanitize () function with your amended code from the previous page, and then 
press F5 in IDLE to confirm the sorting is now working as expected. 


The data 

do»lams as a 


*oo 





Python; Shell 


R.ESTART 


2 

3 

3 

2 


45 

10 

02 

55 


3 

3 

3 

2 


01 

10 

02 

55 


3 

3 

3 

2 


21 

21 

22 

58 


i 

j 

j 

j 


1 f\ 





\ 



Ln: 10 Coi: 4 


UPS 




The oirdeving */orks, bedause ali the 
tir*es are how eomparable. 


Fouir sovTed 

lis-ts 


This output looks much better. 

It 5 s taken a bit of work, but now the data from each of the four files is both 
sorted and uniformly formatted. By preprocessing your data before you 
sort it, you Ve helped ensure Python’s sorting technology performs correctly 



By default, both the sort () method and the sorted () 
BIF order your data in ascending order. To order your data in 
descending order, pass the reverse=True argument to 
either sort () or sorted () and Python will take care of 
things for you. 


you are here ► 


153 

































duplicated code 



Hang on a sec! Some+hing doesn'+ f eel quite 
right. Jook at all that duplicated code, as well as 
all those duplicated lists. This duplication is bad, 
right? Is this really the best you can do? 


That’s right. Duplicated code is a problem. 

As things stand, your code creates four lists to hold the data as read 
from the data files. Then your code creates another four lists to hold 
the sanitized data. And, of course, you’re iterating all over the 

place... 

There has to be a better way to write code like this. 

Transforming lists is such a common requirement that Python 
provides a tool to make the transformation as painless as 
possible. This tool goes by the rather unwieldly name of list 
comprehension. And list comprehensions are designed to reduce 
the amount of code you need to write when transforming one list 
into another. 
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comprehending data 


Comprehending lists 

Gonsider what you need to do when you transform one list into another. Four 
things have to happen. You need to: 


o 

o 

o 

o 


Create a new list to hold the transformed data. 
Iterate each data item in the original list. 

With each iteration, perform the transformation. 
Append the transformed data to the new list. 


|. Cveate- 


clean_mikey = [] 


2- Iterate- 


for each_t in mikey: 






Y Tvans-Covr*. 


clean_mikey.append(sanitize(each t) ) 

Vh —— 




Appchd. 

Here 5 s the same functionality as a list comprehension, which involves 
creating a new list by specifying the transformation that is to be applied to each 
of the data items within an existing list. 


/ou get to piik the target 
'denti-rier to use (just like with 
vegulav iteratiow). 


The new list is ereated-. 



...by applym^ a 
•tvans-fovrination. • • 


..to cacM 
data ite*»... 


...withih ah cxistihj list- 


What 5 s interesting is that the transformation has been reduced to a single line 
of code. Additionally, there 5 s no need to specify the use of the append () 
method as this action is implied within the list comprehension. Neat, eh? 
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idle session 


■ ” / An IDLE Session- 

U-A 

Let's see some other list comprehension examples. Open up your IDLE shell and follow along with these one-liner 
transformations. 


Start by transforming a list of minutes into a list of seconds: 


»> mins = [1, 2, 3] 


»> secs = [m * 
»> secs 
[60, 120, 180] 


60 for m in mins] 



Sinrfly mul-fciply -fche 

values by l>0' 


How about meters into feet? 


»> meters 
»> feet = 
»> feet 


= [1, 10, 3] 

[m * 3.281 for m in meters] 



Yes, -tbeve ave 3-2-®! 
-feet i* ^ Knetev. 


[3.281, 32.81, 9.843] 


Given a list of strings in mixed and lowercase, it's a breeze to transform the strings to UPPERCASE: 

»> lower = ["I", "don't M , "like", "spam"] 

»> upper = [s.upper() for s in lower] 

»> upper -striAJ tomes with 

"DON' T" , 'LIKE', 'SPAM'] the "uppe\rO" method- 

Let's use your sanitize () function to transform some list data into correctly formatted times: 

|t’s never beer, so easy t o W> sometW^ 
dirty rnto sometWmS tlean- © 

lt's also possible to assign the results of the list transformation back onto the original target identifier. This 
example transforms a list of strings into floating point numbers, and then replaces the original list data: 

»> clean = [float(s) for s in clean] 

U ' ' fM0 ' 8 F <«»<* b floati», 

And, of course, the transformation can be a function chain, if that's what you need: 

»> clean = [float(sanitize(t)) for t in ['2-22', '3:33', '4.44']] 

Combmm^ tvanstovmatjons or. tbe data 
items is sw^ovted, too. 


»> clean 
[2.22, 3.33, 4.44] 




»> clean 
[ 2 . 22 , 2 . 22 , 2 . 22 ] 



»> dirty = ['2-22', '2:22', '2.22'] 

»> clean = [sanitize (t) for t in dirty] 
»> clean 

['2.22', '2.22', '2.22'] 
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comprehending data 


^ I 





Now that you know about list comprehensions, let's write four of 
them to process the coach's four lists of timing values.Transform 
each of your lists into sorted, sanitized version of themselves. 
Grab your pencil and in the space provided, scribble the list 
comprehensions you plan to use. 


tfiereiar© no 

Dumb QuestiQns 



So...let me get this straight: list comprehensions are good and list iterations are bad, right? 


No, that’s not the best way to look at it. If you have to perform a transformation on every item in a list, using a list comprehension is the 
way to go, especially when the transformation is easily specified on one line (or as a function chain). List iterations can do everything that list 
comprehensions can, they just take more code, but iterations do provide more flexibility should you need it. 



Python's list comprehension is an example of the language's 
support for functionalprogramming concepts. There's plenty of 
debate about the best way to develop program code: either 
procedurally, using functional programming techniques, or 
using object orientation. At Head First Labs, we try not to get 
involved in this debate, other than to rejoice in the fact that 
Python supports, in one way or another, all three of these 
programming practices. 


you are here ► 


157 


















list comprehensions 



Now that you know about list comprehensions, you were to write 
four of them to process the coaclYs four lists of timing values. You 
were to transform each of your lists into sorted, sanitized version 
of themselves. You were to grab your pencil and in the space 
provided, scribble the list comprehensions you plan to use. 


The list Comprehension 

iri, and the.. ^ d([sa ^ (V . h. iin ) amesj) 

new list is then ordeved 

by the "sovtedO” m 


sovted(Csaniti«(t) -for t in julieJ) 
so\rtcdfCsanitiz^(-t) -fo\r *t in mikeyJ) ' 
so\rted(£sanitiz-e(t) -pov t in sarahJ) 


/?insc and v-epea-fc 
$oy -the o-then lists. 



Watch it! 


Be careful about where you use 
the sorted() BIF when defining 
your list comprehensions. 


You may have been tempted to use the 
function Chain sorted (sanitize (t) ) 

within your list comprehension. Don ’t be. Recall that the 
transformation works on one list item at a time, not the 
entire list. In this example, the sorted () BIF expects 
to sort a list, not an individual data item. 


The beauty of list comprehensions 

The use of list comprehensions with the coaclYs athlete data has resulted 
in a lot less code for you to maintain. Additionally, as you get used to list 
comprehension syntax and usage, youhl find that their use is natural and 
matehes the way your brain thinks about your data and the transformations 
that you might want to apply. 

Let’s confirm that your new code is working as expected. 
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comprehending data 



Replace your list iteration code from earlier with your four new (beautiful) list comprehensions. Run 
your program to confirm that the results have not changed. 


coach4, py - /Users/barryp/HeadFi rstPython/chapterS/coach 4.py 


def sanitize( time string): 
i£ ' - f in time string: 

splitter = '-' 
elif ' : ' in time string: 

splitter * r : r 
else: 

returnftime string) 

(mins, secs) = time string,split(splitter) 
returnfmins + 1 + secs) 


with open( ' james. txt ' ) as 

data = jaf.readline() 
james 


ja£: 



E .L 


T_E_ j .-.l 


i 

Python Shell 
















>» - 

»> 





-- nrisiiirti —- 





[ '2.01 1 , 

'2,01 ' 

f 

'2.22 F , 

'2.34 ' , 

'2,34 ' , 

'2*45 F , 

'3,01 F , 

'3,10 ' , 

'3.21 ' ] 


1 p 2,ir , 

'2,11 ' 

f 

'2.23 F , 

'2.23 F , 

'2,59 F , 

F 3,10 F , 

'3,10 F , 

'3,21', 

'3.21 ' ] 


[ 1 2 * 2 2 1 , 

p 2*38 F 

f 

'2.49', 

'3.01 ' , 

'3.01 F , 

'3,02', 

'3,02', 

'3,02 ' , 

'3.22 ' ] 


[ p 2 * 1S p , 
»> | 

'2,25 ' 

f 

'2,39', 

'2.54 F , 

'2.55', 

'2.55', 

'2.55', 

'2,58' , 

'2.58'] 

A 

T 

juen i sctj.c 

111 . UAL 

) 

as sais 






\ Ln: 10 Coi: 4 

& 


with 

d 

julie 
with 

d 

mikey 

with 

data = saf.readlinef) 
sarah = data,strip().split ( r , 1 ) 

print ( sorted( [ sanitise(t) for t in 
print j sortedj [ sanitizejt) for t in 
print j sortedj [ sanitizejt) for t in 
print j sortedj [ sanitizejt) for t in 


j ames])) 
julie])) 
mikey])) 
sarah])) 



Youv «evx list tom^retensioy>s 
pvodwte t/ACTLY tte " 
same outfwt as youv eavlier 
list itevatio«s. 


Lm: 27 Coi: 0 


A 


As expected, the outout matches that from earlier. 

YouVe written a program that reads Goach Kelly 5 s data from his data files, 
Stores his raw data in lists, sanitizes the data to a uniform format, and then 
sorts and displays the coaclis data on screen. And ali in 25 lines of code. 

It’s probably safe to let the coach take a look at your output now. 

What will the coach think? 
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list slicing 



Have you not been drinking enough 
water? I wanted the three fastest times 
for each a+hle+e...but youve given me 
everything and it contains duplicates! 


In your haste to sanitize and sort your data, you forgot to worry about what 
you were actually supposed to be doing: producing the three fastest times for each 
athlete. And, of course, there 5 s no place for any duplicated times in your 
output. 

Accessing the first three data items from any list is easy. Either specify each list 
item individually using the Standard notation or use a list slice: 


Addess eadt data ite* 
you need 'mdWidually- 



Use a list sl.de to addess tvom list 

item 0 uf-to-but-Kot-mtludmS 

list itcw> 


But...what about removing duplicates from your list? 
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comprehending data 


Iterate to remove duplicates 

Processing a list to remove duplicates is one area where a list comprehension 
cani help you, because duplicate removal is not a transformation; it’s more of 
a filter. And a duplicate removal filter needs to examine the list being created 
as it is being created , which is not possible with a list comprehension. 

To meet this new requirement, youil need to revert to regular list iteration 
code. 



Assume that the fourth from last line of code from your current program is changed to this: 

james = sorted([sanitize(t) for t in james]) 

That is, instead of printing the sanitized and sorted data for James to the screen, this line of 
code replaces James’s unordered and nonuniform data with the sorted, sanitized copy. 

Your next task is to write some code to remove any duplicates from the j ames list produced 
by the preceding line of code. Start by creating a new list called unique_j ames, and then 
populate it with the unique data items found in j ames. Additionally, provide code to display only 
the top three fastest times for James. 

Hint: you might want to consider using the not in operator. 
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top three 



ExwctSe 


§0(.|#tl0H 


Assume that the fourth from last line of code from your current program is changed to this: 

james = sorted([sanitize(t) for t in james]) 

That is, instead of printing the sanitized and sorted data for James to the screen, this line of 
code replaces James’s unordered and non-uniform data with the sorted, sanitized copy. 

Your next task was to write some code to remove any duplicates from the j ames list produced 
by the preceding line of code. You were to start by creating a new list called unique_j ames 
and then populate it with the unique data items found in j ames. Additionally, you were to 
provide code to only display the top three fastest times for James. 


Cv-eate the 
e**pty list to 
hold the *ni<\ue 
data ite**s. 


u»i<\ue_jar*es — [J 

✓-/ternate ovev- -the 

r ^ CX ist i na dati... n 

tov eadh_t in James: ...and it the data item |£/\TT 

. p aliready in the ne*/ list... 

i-r eadh_t not in <*ni^ue__James: 

uni^ue_jar^e s append(ea^h_t) :v a ?? cyy ^ 

~ the ne*/ list- 


Slide the tinst 

three data ite ms . 

tnom the list and print(uni^ue_james[0:3J) 

display them on 

s£reen. 



Repeat the code on this page 
for the rest of the coacffs lists: 
julie, mikey & sarah. Add 
all of your new code to your 
existing program. 
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comprehending data 



Take ali of the recent amendments and apply them to your program. Run this latest code within IDLE 
when you are ready. 


^ ^ coach5.py - /Users/barryp/HeadFirstPython/chapter5/coach5.py 


james = sorted( [sanitize(t) for t in james]) 
julie = sorted( [sanitize(t) for t in julie]) 
mikey = sorted( [sanitize(t) for t in mikey]) 
sarah = sorted( [sanitize(t) for t in sarah]) 



Solrt and sani-fciz* 


unique james = [] 
for each t in james: 

if each_t not in unique james: 
unique_ j ames.append(eacht) 

print (unique_ j ames[0:3]) 

unique julie = [] 
for each t in julie: 

if each t not in unique julie: 
unique julie.append(each_t) 

print (unique_julie[0:3]) 

unique mikey = [] 
for each t in mikey: 

if each t not in uniquemikey: 
uniquemikey.append(eacht) 




ea£h lis-fc. 


Remove 


print (uniquemikey[0:3]) 


unique sarah = [] 
for each t _ . .. sarah: 


*oo 


Python Shell 


pri 


»> 

»> 

[ '2 

[ 


RESTART 


2 
2 
[ F 2 
>» 


[ 


01 

11 

22 

18 


2.22 

2.23 

2.38 

2.25 


2 « 3 4 p ] 
2.59 p ] 
2.49 p ] 
2.39 1 ] 


Lookihg good^ 


0 



It worked! 

You are now displaying only the top three times for each athlete, and the 
duplicates have been successfully removed. 

The list iteration code is what you need in this instance. There 5 s a little bit of 
duplication in your code, but it’s not too bad, is it? 
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duplicated code to remove duplication? 



The irony is hard to avoid, sn’t it? 

The code that removes duplicates from your lists is itself 
duplicated. 

Sometimes such a situation is unavoidable, and sometimes 
creating a small function to factor out the duplicated code can 
help. But something stili doesn’t feel quite right here... 
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comprehending data 


Wouldn't it be dreamy if there were a way to 
quickly and easily remove duplica+es from an 
existing list? But I know it 's just a fantasy... 
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factory functions 


Remove duplicates with sets 

In addition to lists, Python also comes with the set data structure, which 
behaves like the sets you learned ali about in math class. 

The overriding characteristics of sets in Python are that the data items in a set 
are unordered and duplicates are not allowed. If you try to add a data item to a set 
that already contains the data item, Python simply ignores it. 

Create an empty set using the set () BIF, which is an example of a factory 
functiorv. 

Clreate 3 hCW; 
cwpty se-t, and 
n it io a 
variable. 



distances = set() 


It is also possible to create and populate a set in one step. You can provide a list 
of data values between curly braces or specify an existing list as an argument 
to the set () BIF, which is the factory function : 



A*y duplieates i» 
the supplicd list 
o-f data values av~e 
•gnoved. 


duplicates 

the u jar»es list av-e 
i<yr\oved- Cool. 





Fa^tov-y Fuhctioh* f\ -factavy -Puncti ok\ is used “to r*ake kcw 
iicms o-f a parti£ulav type. Fov- "sctO” is 

'PadWy -fuhdtioh bedause i“t makes a ^cv/ se-t. I* “tbe 
rcal v/orld, ^^“tories r*ake “tbrn^s, bchdc “tbc ^a^c- 
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comprehending data 


Fireside Chats 



List: 

[sings] “Anything you can do, I can do better. I can 
do anything better than you.” 

Can you spell “d-a-t-a 1-o-s-s”? Getting rid of data 
automatically sounds kinda dangerous to me. 

Seriously? 


And that’s all you do? 

And they pay you for that?!? 


Have you ever considered that I like my duplicate 
values. Fm very fond of them, you know. 

Which isn 5 t very often. And, anyway, I can always 
rely on the kindness of others to help me out with 
any duplicates that I don’t need. 


Set: 

Fm resisting the urge to say, “No, you can 5 t.” 
Instead, let me ask you: what about handling 
duplicates? When I see them, I throw them away 
automatically. 

But that 5 s what Fm supposed to do. Sets aren’t 
allowed duplicate values. 

Yes. ThaFs why I exist.. .to sto re sets of values. 
Which, when it’s needed, is a real lifesaver. 

Thafs all I need to do. 

Very funny. You’re just being smug in an effort 
to hide from the fact that you can’t get rid of 
duplicates on your own. 


Yeah, right. Except when you dordt need them. 

I think you meant to say, “the kindness of set () ”, 
didn’t you? 



To extract the data you need, replace 
all of that list iteration code in your 
current program with four calls to 
sorted(set(...)) [0:3]. 
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code review 




ChJe Ppvjew 


The Head First Code Review Team has taken your code and 
annotated it in the only way they know how: they’ve scribbled 
all over it. Some of their comments are confirmations of what 
you might already know. Others are suggestions that might 
make your code better. Like all code reviews, these comments 
are an attempt to improve the quality of your code. 



def sanitize(time string) 




if in time string: 

splitter = '-' 
elif ':' in time string: 

splitter = ':' 
else: 

return(time string) 

(mins, secs) = time string.split(splitter) 
return(mins + ' . ' + secs) 

with open ('james.txt') as jaf: 


A Would 

be hi£e io have 
here. 


o 



1 


data = jaf.readline() 
james = data.strip ().split (',') 


What V>ayv e ' r ' s ^ 

•Jf o*e of these 

?i? 

Jf\les »s . • 

, ^-^ 

WV^ere S with open ( 'julie.txt ' ) as juf: 

e*teption bandl»n$ data = juf.readline() 

■" \ .. 

with open( 'n^^ey^txt') as mif: 

data = mif.readline() 
mikey = data.strip().split(',') 

with open('sarah.txt') as saf: 

data = saf.readline() 
sarah = data.strip().split(',') J 

print (sorted(set([sanitize(t) for 
print(sorted(set([sanitize(t) for 
print(sorted(set([sanitize(t) for 
ead Vt f' ron ' tV ' e print(sorted(set([sanitize(t) for 

msidc OVA *b' 


II 


Meet t* Head 
pivst Code Review 

Tcd**- 




There s a lot 

Wt Nve W !' s 
*ot -boo '' a ' r< ^ 

vdevstaind \f 1°» 


Theres a bit of duplitatior, here- Y°u 

tould -fattor out the tode i*to a sw<a " 
UU the«, all 70« meed to do is tali 

the Wti» f&r eath of 7«*v athlete 
data -files, assiy^ the resuit to an 
athlete list 



V" 


t in j ames])) [0:3]) 
t in julie])) [0:3]) 
t in mikey])) [0:3]) 
t in sarah])) [0:3]) 



Ah, OK. We get it- 

The slide is applied to 
the list produded bv 
'WtedO" v-i^h-t? 


L 
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comprehending data 



Let’s take a few moments to implement the review team’s suggestion to turn those four with 
statements into a function. Here’s the code again. In the space provided, create a function to 
abstract the required functionality, and then provide one example of how you would call your 
new function in your code: 


with open('james.txt') as jaf: 

data = jaf.readline() 
james = data.strip () .split ( ', ') 


with open('julie.txt') as juf: 

data = juf.readline() 
julie = data.strip().split) 


with open('mikey.txt') as mif: 

data = mif.readline() 
mikey = data.strip().split (',') 


with open('sarah.txt') as saf: 

data = saf.readline() 
sarah = data.strip().split) 


VVv-i-bc yvcvi 

-functio* heve. 



Provide ohe 
example dall. 
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statement to function 



SoLutiort 


You were to take a few moments to implement the review team’s suggestion to turn those four 
with statements into a function. In the space provided, your were to create a function to 
abstract the required functionality, then provide one example of how you would call your new 
function in your code: 


with open ('james.txt') as jaf: 

data = jaf.readline() 
james = data.strip ().split ) 


with open('julie.txt') as juf: 

data = juf.readline() 
julie = data.strip().split) 


with open('mikey.txt') as mif: 

data = mif.readline() 
mikey = data.strip().split) 



with open ('sarah.txt') as saf: 

data = saf.readline() 
sarah = data.strip().split) 

- 3 f ilcy)3mc 3s 

, r i . , i /ri ^ ^ so * c 

^ ^ dt\ <\et toacM dataC+ilchamcy: 

Cvcatc a 3 — — 

hnLhon' ." . 

' ^-—— ^ “bhe 

y/i*th opch^ilcharwc) as «f: v-cad ite da^ca- 

ewefW-hahaiihj . d . ab .".^:' rCad me0 . Pe^-the spliVstHp Wk o. 

ve-ti*v-h(data stv-ip0.splii('/)) 1^'°a ' re ^ l> ' ri,lh 9 to 

e%dep*t I0&rv-o\r as ioev-v-: 

pv-mtCFilc c\r\ro\r: 1 + s*br(ioe\r\r)) 

vetuvhMhe) fili \io uv iascv- about tVic ep o,r ^ 

(tf it otdurs) a»d vetwv-h "NonC 
to mditate PaiWe- 

Callis the Puhfctioh sarah — jet^oadl^dataPsaraht*!) 
is stvai^t^^avd- 

Provide the ha^e oP the -Pile to 

pv-o£ess. 
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comprehending data 



lt's time for one last run of your program to confirm that your use of sets produces the same results 
as your list-iteration code. Take your code for a spin in IDLE and see what happens. 


coach 5 . py - /Users/barryp/HeadFi rstPython/chapterS /coach 5 . py 

def sanitise( time string): 
if r - f in time string: 

splitter = F - F 
elif F : F in time string: 

splitter = F : r 
else: 

return(time string) 

(mins, secs) = time string,split(splitter) 
retarn(mins + F . 1 + secs) 

with open( 1 james . txt 1 ) as jaf: 

data = jaf , readline() 
james = data.strip()>split( 1 # 1 ) 




Excellent! 



YouVe processed the coach 5 s data perfectly, while 
taking advantage of the sorted () BIF, sets, 
and list comprehensions. As you can imagine, you 
can apply these techniques to many different 
situations. You’re well on your way to becoming 
Python data-munging masterl 
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python toolbox 



Your Pythow Toolbox 

You’ve got Chapter 5 under your 
belt and you’ve added some more 
Python techiques to your toolbox. 


ar ^ » „ re ad'^ 

t J -«^ s *” dsb 

'JW * data 

toWetW oJf 


^ p yiho» u 

* wtl? W - T«' 

"s-ro^tioh oh ohe Ii** (, 

^ <» »» 5 * 

® A slicc _ ad< , css ^ 

,te * ^ a list tha * c 

* ^ J e ^ ~ a tolletlio» o$ 
CwaJ^ da ^ i ^ ems Otft 

* h,M »° dupli^. 



BULLET 


POINTS 


■ The sort () method changes the 
ordering of lists in-place. 


■ The sorted () BIF sorts most any data 
structure by providing copied sorting. 


■ Pass reverse=True to either 

sort () or sorted () to arrange your 
data in descending order. 


■ When you have code like this: 

new 1 = [] 
for t in old 1: 
new 1. 
append(len(t)) 

rewrite it to use a list comprehension, 

like this: 

new 1 = [len(t) for t 
in old 1] 


■ To access more than one data item from 
a list, use a slice. For example: 

my_lis t[3:6] 

accesses the items from index location 3 
up-to-but-not-including index location 6. 


■ Create a set using the set () factory 
function. 
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6 ciistom dcttA objects 




lt’s important to match your data structure choice to your data. 

And that choice can make a big difference to the complexity of your code. In Python, 
although really useful, lists and sets aren’t the only game in town. The Python dictionary 
lets you organize your data for speedy lookup by associating your data with names, not 
numbers. And when Python’s built-in data structures don’t quite cut it, the Python class 
statement lets you define your own. This chapter shows you how. 


this is a new chapter 
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additional data 


Coach Kelly is back 
(with a wew file format) 



I love what youve done, but I can'+ teli which line 
of data belongs to which a+hle+e, so IVe added some 
Information to my data files to make it easy for you to 
f igure it out. I hope this doesn'+ mess things up much. 


The output from your last program in Ghapter 5 was exactly what the coach 
was looking for, but for the fact that no one can teli which athlete belongs to 
which data. Coach Kelly thinks he has the solution: he 5 s added identification 
data to each of his data files: 


This is "sarah2--t*t", 'M | th 

e*tra data added- 



Sarah s -full Sav-alVs da*tc ©f biv*th 


Sav-aWs tWmj da-ta 


If you use the split () BIF to extract Sarah’s data into a list, the first data 
item is SaralTs name, the second is her date of birth, and the rest is SaralTs 
timing data. 

Let’s exploit this format and see how well things work. 





Grab the updated files from the 
Head First Python website. 
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custom data objects 



CoJe Magnets 


Let's look at the code to implement the strategy outlined at the bottom of the previous page. For 
now, let's concentrate on Sarah's data. Rearrange the code magnets at the bottom of this page to 
implement the list processing required to extract and process Sarah's three fastest times from Coach 
Kelly's raw data. 


Hint: the pop () method removes and returns a data item from the specified list location. 


def sanitize(time string): 
if in time string: 

splitter = '-' 
elif ':' in time string: 

splitter = ':' 
else: 

return(time string) 

(mins, secs) = time string.split(splitter) 
return(mins + '.' + secs) 



def get_coach_data(filename): - The "oye-t__£oath__da-taO" Wbon IS 

try: also -fv-om -the last thaftev- 

with open (filename) as f: 

data = f.readline() 
return(data.strip() .split (',')) 
except IOError as ioerr: 

print('File error: ' + str(ioerr)) 
return(None) 


Rearvan^e ihe 

rr>ay>eis here- 
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sarah’s times 



Cocte Magnets Solution 

Let's look at the code to implement the strategy outlined earlier. For now, let's concentrate on 
Sarah's data. 

You were to rearrange the code magnets at the bottom of the previous page to implement the list 
Processing required to extract and process Sarah's three fastest times from Coach Kelly's raw data. 


def sanitize(time string): 
if 'in time string: 

splitter = '-' 
elif 1 :' in time string: 

splitter = ':' 
else: 

return(time string) 

(mins, secs) = time string.split(splitter) 
return(mins + '.' + secs) 


def get_coach_data(filename): 
try: 

with open(filename) as f: 

data = f.readline() 
return(data.strip () .split (',')) 
except IOError as ioerr: 

print('File error: ' + str(ioerr)) 
return(None) 


r 



Wse the -funition to twvn 
Sa^ahs data -Pile i„to a list, 
a»d the» ass ig» it to the 
sardh vaviable. 


sarah.pop(0), sarah.pop(0) 



The V ? (0)" tal ' I Print(sarah_name + | 

vetuv^s a*\d 
«-emoves data Prom 
the -fvont of a 
list- Two talis to 

veroove 

tbe &v-st two data 

valucs a*d asst^s 
tbe** to tbc y\aw>cd 
vaviables. 


A ^ustom mcssagc withih 
the tali to V-mtO" is used 
to display the vesJts youVe 

a+tcir. 
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custom data objects 



Let's run this code in IDLE and see what happens. 


O 


coath2. py - /Users/barryp/HeadFi rst Python /chapter6/coach2. py 


def sanitise) time string): 
if 1 - 1 in time string: 

splitter = 1 - ’ 
elif F : F in time string: 

splitter = ' : F 
eis e : 

returnftime string) 

(mins, secs) = time string.split ( splitter) 
return(mins + 1 . 1 + secs) 

def get coach data ( filename): 
try : 

with open(f ilename) as f: 

data = f.readline)) 
return) data.strip)).split) 1 , 1 )) 
except IOError as ioerr: 

print('File error: 1 + str(ioerr)) 
return(None) 

sarah = get_coach_data) 1 sarah2,txt 1 ) 

(sarah name, sarah dob) = sarah.pop)0) f sarah.pop(Q) 

print(sarah name + 11 1 s fastest times are: 1,1 + 

str ( sorted)set ( [ sanitise)t) for t in sarah ] )) [ 0:3 ] )) 



Youv la-fces-fc ^ oc | c 



f 


This 

is wudh w\OVC 
wndierstawdiable- 




Python Shell 


Python 3.1.2 (r312:7936GM, Mar 24 2010, 01:33:18} 

[GCC 4.0.1 (Apple Inc. build 5493)] on darwin 

Type 11 Copyright H , '"credits" or 11 licens e) ) " for more information. 
»> ================================ RESTART =================== 

»> 

Sarah Sweeney F s fastest times are: [ F 2.18 F , F 2.21 F , 1 2,22 F ] 

»> I 


Ln: 7 Cof: 4 


This program works as expected, and is fine.. .except that you have to name and create 
Sarah’s three variables in such as way that it’s possible to identify which name, date of birth, 
and timing data relate to Sarah. And if you add code to process the data for James, Julie, 
and Mikey, youil be up to 12 variables that need juggling. This just about works for now 
with four athletes. But what if there are 40, 400, or 4,000 athletes to process? 

Although the data is related in “real life,” within your code things are disjointed, because 
the three related pieces of data representing Sarah are stored in three separate variables. 
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keys and values 


The 


Use a dictiowary to associate data 

Lists are great , but they are not always the best data structure for every 
situation. Let 5 s take another look at Saraffs data: 


SaraWs -Culi s date o£ bi\rfch 


Samah s timihg data 


Sarah Sweeney,2002-6-17,2:58,2.58,2:39,2-25,2-55,2:54,2.18,2:55,2:55,2:22,2-21,2.22 


There’s a definite structure here: the athlete 5 s name, the date of birth, and 
then the list of times. 

Let 5 s continue to use a list for the timing data, because that stili makes sense. 
But let 5 s make the timing data part of another data structure, which associates 
all the data for an athlete with a single variable. 

We’ll use a Python dictionary, which associates data values with keys : 



"Sarah Sweeney" 



"2002-6-17" 



The assodiated data, ako k*ow* 
as the "values" 



[2:58,2.58,2:39,2-25,2-55,2:54,2.18,2:55,2:55,2:22,2-21,2.22] 




Did“tio*ary A buil“t —m data strudture (mdluded witb 
Pytho*) that allov/s you to assodiate data v/ith keys, as 
opposed to *umbers. This lc*bs your in—memor y da*ta 
dlosely matdh the strudture ot your adtual da*ta. 
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custom data objects 


Fireside Chats 



Dictionary: 


Tonightfs talk: To use a list or not to use a list? 


List: 


Hi there, List. I hear you’re great, but not always 
the best option for complex data. That’s where I 
come in. 


What?!? Haven’t you heard? You can put anything 
into a list, anything at ali. 


True. But when you do, you lose any structure 
associated with the data you are processing. 


Isn’t it always? 


Well.. .assuming, of course, that structure is 
important to you. 

Ummm, uh.. .1 guess so. 


You guess so? When it comes to modeling your data 
in code, it 5 s best not to guess. Be firm. Be strong. Be 

assertive. Use a dictionary. That sounds like a slogan from one of those awful 

self-help conferences. Is that where you heard it? 


[laughs] Oh, I do love your humor, List, even when 
you know you 5 re on thin ice. Look, the rule is 
simple: if your data has structure, use a dictionary, not a 

list. How hard is that? , T . . . „ TT . r 

JNot that hard, really. Unless, ol course, you are a 

list, and you miss being used for every piece of data 

in a program... 


Which rarely makes sense. Knowing when to use a 
list and when to use a dictionary is what separates 
the good programmers from the great ones, right? 

I guess so. Man, I do hate it when you’re right! 



The Python dictionary is known by different names in other programming languages. If you hear other 
programmers talking about a "mapping,"a "hash,"or an "associative array,"they are talking about a "dictionary." 
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idle session 


■ ” / An IDLE Session- 

U-/ 

Let's see the Python dictionary in action. Follow along with this IDLE session on your computer, ensuring that you 
get the same results as shown. 

Start by creating two empty dictionaries, one using curly braces and the other using a factory function: 


»> cleese = {} 
»> palin = dict () 
»> type (cleese) 




Bo-fch tedhui^ues dveate 
2 * Cr^p-ty didtionav-y, as 
^oh-riirirhcd- 


Add some data to both of these dictionaries by associating values with keys. Note the actual structure of the data 
is presenting itself here, as each dictionary has a Name and a list of Occupatioris. Note also that the palin 
dictionary is being created at the same time: 

»> cleese [' Name' ] = ' John Cleese' 

»> cleese['Occupations'] = ['actor', 'comedian', 'writer', 'film producer'] 

»> palin = {'Name': 'Michael Palin', 'Occupations': ['comedian', 'actor', 'writer', 'tv']} 


With your data associated with keys (which are strings, in this case), it is possible to access an individual data item 
using a notation similar to that used with lists: 

Use s^uare bvadkets to index into tbe didtionavy to addess 
data items, but instead o-f numbev-s, index witb keys. 

Use numbevs io addess a list item stoved at a pavtidulav didWav-y key- 
TVmk of this as w mdey.-tba'm'm9" and v-ead fv-om vigbt to leffc "...tte last 
item of tte list assodiated wtb Ottu^ations...’. 

As with lists, a Python dictionary can grow dynamically to store additional key/value pairings. Let's add some data 

about birthplace to each dictionary: _ _ , 

^ nrovide tbe data assodiated 

»> palin ['Birthplace' ] = "Broomhill, Sheffield, England" witb tbe new key. 

»> cleese['Birthplace'] = "Weston-super-Mare, North Somerset, England" 


»> palin [' Name' ] ^- 

'Michael Palin' 

»> cleese [ 'Occupations ' ] [-1] 
'film producer' 


Unlike lists, a Python dictionary does not maintain insertion order, which can resuit in some unexpected 
behavior. The key point is that the dictionary maintoins the associations , not the ordering: 

»> palin 

{'Birthplace': 'Broomhill, Sheffield, England', 'Name': 'Michael Palin', 'Occupations': 

['comedian', 'actor', 'writer', 'tv']} 

»> cleese 

{'Birthplace': 'Weston-super-Mare, North Somerset, England', 'Name': 'John Cleese', 
'Occupations': ['actor', 'comedian', 'writer', 'film producer']} 

Tbe ov-devmg r*a'mta'med by Pytbon is dicerent £vom bov/ tbe data 
was insevted- Don*t wovvy about »tj tbis is 0^- 
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custom data objects 



lt’s time to apply what you now know about Python’s dictionary to your code. Let’s continue to 
concentrate on Sarah’s data for now. Strike out the code that you no longer need and replace it 
with new code that uses a dictionary to hold and process Sarah’s data. 

def sanitize(time_string): 
if in time_string: 

splitter = '-' 
elif ':’ in time_string: 

splitter = ' : ' 
else: 

return(time_string) 

(mins, secs) = time_string.split(splitter) 
return(mins + '.' + secs) 


def get_coach_data(filename): 
try: 

with open(filename) as f: 

data = f.readline() 
return(data.strip () .split ( ', ' ) ) 
except IOError as ioerr: 

rL print('File error: ' + str(ioerr)) 

i»t\rike out -the tode . , 

return(None) 

sarah = get_coach_data(’sarah2.txt') 

(sarah_name, sarah_dob) = sarah.pop(0), sarah.pop(O) 
print(sarah_name + M, s fastest times are: M + 

, str(sorted(set([sanitize (t) for t in sarah])) [0:3])) 

/\dd Youv d^tto^avY 

us'm^ av\d prodcssmj . 

dodc 
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dictionary data 



ExeftciSe 


Solitiori 


lt’s time to apply what you now know about Pythorfs dictionary to your code. Let's continue to 
concentrate on Sarah's data for now. You were to strike out the code that you no longer needed 
and replace it with new code that uses a dictionary to hold and process Sarah’s data. 

def sanitize(time_string): 
if in time_string: 

splitter = '-' 
elif ':' in time_string: 

splitter = ':' 
else: 

return(time_string) 

(mins, secs) = time_string.split(splitter) 
return(mins + '.' + secs) 


def get_coach_data(filename): 
try: 

with open(filename) as f: 

data = f.readline() 
return(data.strip() .split (',')) 
except IOError as ioerr: 

print('File error: ' + str (ioerr)) 
return(None) 


Y°u doni need -this 
£°de dhyr»ov~e. 



P r i |~1 I f^M ll IldlllB 


Cveate a» e»>pty 
dit-tionav-y. 


- " ’ tasi.es i. LiniLj-a^:—-— 

- 1 1 Nin n 1-nHfnnf/[mni1-i7 <=rp ) f ■ e in - - t ~ b ] M [l 1 


sarah data - {} 


sa»rah_dataf Ka»»c'3 - sarah.popfO/ _ Populate the d^tio^y with the daU u Y 

sav-ah dataf POB'3 = sa^ahf op^^^ ~ the data 4Vom the fi| e with Y 

. 7“. ..... 1 .. T . .7j.T*?. dittiortry keys.. 

sav-ah__da-taCTimes3 =■ sav-ah 

p\r‘m-t(sarah_da-tarName*3 + u; s -Pas-tes-t -times are w + 

P s-tv-^sov-ted^se-t^Csa^i-tiz^f-t) £ov- -t m sav-ah da-tarTimes ; 33))CO:33)) 

Ke-Pev -to -the- die-tibhaVy wheh. 

pv-oeessihj -the da-ta. —- 
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custom data objects 



Let's confirm that this new version of your code works exactly as before by testing your code within 
the IDLE environment. 



Which, again, works as expected.. .the difference being that you can now more easily 
determine and control which identification data associates with which timing data, 
because they are stored in a single dictionary. 

Although, to be honest, it does take more code, which is a bit of a bummer. Sometimes the 
extra code is worth it, and sometimes it isn’t. In this case, it most likely is. 

Let’s review your code to see if we can improve anything. 
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code review 



HeaJ flrst 
CoJe Review 


The Head First Code Review Team has been at it 
again: they’ve scribbled all over your code. Some 
of their comments are confirmatioris; others are 
suggestions. Like all code reviews, these comments 
are an attempt to improve the quality of your code. 

def sanitize(time string): 
if in time string: 

splitter = '-' 
elif 1 : 1 in time string: 

splitter = ':' 
else: 

return(time string) 

(mins, secs) = time string.split(splitter) 
return(mins + '.' + secs) 


def get_coach_data(filename): 
try: 

with open(filename) as f 
data = f.readline() 
return(data.strip() .split (', ')) 



n 



except IOError as ioerr: 

print('File error: ' + str (ioerr)) 
return(None) 

sarah = get_coach_data('sarah2.txt') 
sarah_data = {} 

sarah data['Name'] = sarah.pop(O) 
sarah_data['DOB'] = sarah.pop(O) 

;arah data['Times'] = sarah 

print(sarah data['Name'] + M, s fastest times are: 


Rate te w.ia-5 as r 50 

alonft why nat do it all m_ont 50 ? I» W, 
this situate, it ^t tven »ake sense to do 
tw.s ^otessin^ witWm t he 3 et_toath_data 
•fwnttion and have the Wtion «W a 
?0?w lated dittionar/ as o^osed to a l.st 

Then, all yo* need to do is tveate the 
dittionav-y from the data hle usm^ an 
a?? ro ? v-iate wtion tali, ^ht? 


+ 


^str (sorted(set([sanitize(t) for t in sarah_data['Times']])) [0:3])) 

T 

Vou might want to tonsider moving this Code into the get_toath_dataO Wtion, too, betause doing 

would rate nitely abstet away these protessing details. But y/hethev you do «- not is uP to vou- |t’s 
youv- Code, a-P-tev- all/ 


so 
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custom data objects 



Actually, those review comments are really useful. Let's take the time to 
apply them to your code. There are four suggestions that you need to 
adjust your code to support: 

1. Create the dictionary all in one go. 

2. Move the dictionary creation code into the get_coach_data () 
function, returning a dictionary as opposed to a list. 

3. Move the code that determines the top three times for each athlete 
into the get_coach_data () function. 

4. Adjust the invocations within the main code to the new version of the 
get_coach_data () function to support it's new mode of operation. 

Grab your pencil and write your new get_coach data () function 
in the space provided below. Provide the four caIis that you'd make to 
process the data for each of the athletes and provide four amended 
print() statements: 
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reviews are in 



You were to take the time to apply the code review comments to your 
code. There were four suggestions that you needed to adjust your code 
to support: 

1. Create the dictionary all in one go. 

2. Move the dictionary creation code into the get_coach_data () 
function, returning a dictionary as opposed to a list. 

3. Move the code that determines the top three times for each athlete 
into the get_coach_data () function. 

4. Adjust the invocations within the main code to the new version of the 
get_coach_data () function to support its new mode of operation. 

You were to grab your pencil and write your new get_coach data () 
function in the space provided below, as well as provide the four calls 
that you'd make to process the data for each of the athletes and provide 
four amended print () statements: 


de-P jet_fcoadh__datatPileKame): 



with as -(*.• 


data =■ ‘f.readlincO 


| Cveate a tempov-av-y ;>templ — data stvipO.split^/) 

list to hold tVic data , f 

BftFORfc dveatm^ the Name • templ.popCOJ, 

ditWry all rn y>. : icrrylyoyCO), 



2- The ditijondry dreatior, toAt is 
now P^t of the fuhfiti 


;ioh. 


Times 1 : stv"^so\rted^sct^Csar\itiz^(t) -for t m templJ))CO'3J)}) 


^ * iom :: . TWdode that deWmes the 

tthvee stoves is \>avt of the 
-funfction, too. 

retu\rr\(/Vor\e) 


pv-mt^Pile ev-v-ov*: 1 + stv(ioevr)) 


/p. Call the WW "" m n i in ^ S ^ ow,h 9 thesc two l‘mcs ot 

War.attkl* awl = 5 «t_£M£h_data() «T'°de +» «eathlete(beta.* «paU* , t 

tt. >»«)' T. tte ^ « a W,al 

statew>ent as *>eeded- .. 

^ pvmtfjamesC'Na nc3 + u; s -fastest times av-e : w + jamesCTimesT 


h 9 

exemeise). 
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custom data objects 



Let’s confirm that all of the re-factoring suggestions from the Head First Gode Review Team are 
working as expected. Load your code into IDLE and take it for a spin. 


coach 3c.py - ! Users/ barryp/Head FirstPython/chapter6/coach3c.py 


def sanitize( time string): 
if ' - r in time string: 

splitter = ' - ' 
elif ' : ' in time string: 

splitter = 1 : r 
else: 

returnftime string) 

(mins, secs) = time string,split(splitter) 
return(mins + 1 + secs) 



AII o-P -the da-fca pyodcssihg is 
">oved ihto the -fuhdtioh. 


def get coach data( £ilename): 
try : 

with open( filename) as f: 

data = £.readline() 
templ - data.strip().split( 1 f 1 ) 
return( { 1 Name ' : templ.popj0), 

1 DQB ' : templ.popj0) f 

'Times 1 : str ( sorted(set ( [ sanitize(t) for t in templ] ))[ G:3])}) 
except lOError as ioerr: 

print ( 1 File error: r + str (ioerr)) 
return(None) 



james = get_coach data( 1 james2,txt 1 ) 
julie = get_coach dataj ' julie2,txt 1 ) 
mikey = get_coach_data( 'mikey2.txt' } 
sarah = get__coach data j 1 sarah2 , txt 1 ) 


TWis todt bas bee* tottsidevabl7 *bid»cd 
l his c.oac nd Uulc-tc assodia-ted 

yvovi displays -tbe <h tbe athict 

Yjrtb 'tbciv -tWcs. 



print (james [ 1 Name 1 ] + 
print fjulie [ 1 Name 1 ] + 
print fmikey [ 1 Name 1 ] + 
print jsarah[ 1 Name 1 ] + 


ii p 


n p 


s fastest times 
s fastest times 
s fastest times 
s fastest times 


are: 

are: 

are: 

are: 


t j ames [ 1 Times 1 ]) 
t julie [ 1 Times 1 ]) 
t mikey [ 1 Times 1 ]) 
t sarah [ F Times 1 ]) 


^OO 


Python Shell 


’ 

Uookm^ 
aood! 


»> ============================== 

»> 

James Lee r s fastest times are: ['2 
Julie Jones F s fastest times are: ( 
Mikey McManus ' s fastest times are: 
Sarah Sweeney ' s fastest times are: 
»> I 


RESTART 


01 ', '. 
2 , 11 ', 

[ ' 2*22 
['2,18 


16', ' 
2.23 ' , 
'2.31 
' 2.21 


22 ' ] 

2.59 ' ] 
'2.38' ] 
' 2,22 ' ] 


d 


Ln: 19 Coi: 4 ^ 


To process additional athletes, all you need is two lines of code: the first invokes 
the get coach data () function and the second invokes print () . 


And if you require additional functionality, it 5 s no big deal to write more 
functions to provide the required functionality, is it? 
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associate custom code with custom data 


O 



Wait a minute...you're using a dictionary to 
keep your data ali in one place, but now you're 
proposing to write a bunch of custom functions 
that work on your data but aren‘t associated with 
it. Does that really make sense? 


Keeping your code and its data together is good. 

It does indeed make sense to try and associate the functions with the 
data they are meant to work on, doesn’t it? After all, the functions 
are only going to make sense when related to the data—that is, the 
functions will be specific to the data, not general purpose. Because this 
is the case, it 5 s a great idea to try and bundle the code with its data. 

But how? Is there an easy way to associate custom code, in the form 
of functions, with your custom data? 
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custom data objects 


Pundle your code and its data in a class 

Like the majority of other modern programming languages, Python lets you 
create and define an object-oriented class that can be used to associate code 
with the data that it operates on. 




Using a class helps reduce complexity. 

By associating your code with the data it works on, you reduce 
complexity as your code base grows. 



Reduced complexity means fewer bugs. 

Reducing complexity results in fewer bugs in your code. 
However, it 5 s a fact of life that your programs will have 
functionality added over time, which will resuit in additional 
complexity Using classes to manage this complexity is a very 
good thing. 




Fewer bugs means more maintainable code. 

Using classes lets you keep your code and your data together in 
one place, and as your code base grows, this really can make 
quite a difference. Especially when it 5 s 4 AM and you 5 re under a 
deadline... 
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get some class 


Pefi m a class 

Python follows the Standard object-oriented programming model of 
providing a means for you to define the code and the data it works on as a 
class. Once this definition is in place, you can use it to create (or instantiate ) 
data objects, which inherit their characteristics from your class. 

Within the object-oriented world, your code is often referred to as the class 5 s 
methods, and your data is often referred to as its attributes. Instantiated 
data objects are often referred to as instances. 


The w vW’ 



"Sarah Sweeney","2002-6-17" 


[2:58,2.58,2:39,2-25,2-55,2:54,2.18,2:55,2:55,2:22,2-21,2.22] 




The -Pac-fcovy has beer, 
pv-irwed wi-th youv class. 


/VJikey s objed-fc irs-tarCe 


Julius objedt 
ins-tande 


Ja**es s o 


stande 


m 


e ave youv msta^a-ted objeib, 
th ave yatka^ed -to to*vtam youv 
le and i-b assodiaied da-ta- 




Each object is created from the class and shares a similar set of 
characteristics. The methods (your code) are the same in each instance, but 
each object 5 s attributes (your data) differ because they were created from your 
raw data. 


Let’s look at how classes are defined in Python. 
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custom data objects 


Use class to define classes 

Python uses class to create objects. Every defined class has a special method 
called_init_(), which allows you to control how objects are initialized. 

Methods within your class are defined in much the same way as functions, 
that is, using def . Here’s the basic form: 



Creatiwg object instances 

With the class in place, ifs easy to create object instances. Simply assign a call 
to the class name to each of your variables. In this way, the class (together 

with the_init_() method) provides a mechanism that lets you create 

a custom factory function that you can use to create as many object 
instances as you require: 



Unlike in C ++-inspired languages, Python has no notion of defining a 
constructor called “new.” Python does object contruction for you, and then 
lets you customize your objects initial state using the_init_() method. 
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note to self 


The iwportance of sejf 

To confirm: when you define a class you are, in effect, defining a custom factory 
function that you can then use in your code to create instances: 


The idwfcvftar ^hat ^°'^ s 
a v-c-fcvc^dc b> 7ouv msiante 


a = Athlete() 


When Python processes this line of code, it turns the factory function call into 
the following call, which identifies the class , the method (which is automatically 
set to_init_ ()), and the object instance being operated on: 


The harwe o( -the elass 




Uok e the classs cusU. facfa» 
TUhexioh. 


V-y 


Athlete(). init (a) 


Now take another look at how the init () method was defined in the 
class: 


i_The tavjet idcwb-f »cr 

o-f -the objee-t msta^ee 



def _init_(self) : 

# The code to initialize an "Athlete" object. 


Gheck out what Python turns your object creation invocation into. Notice 
anything? 


The target identifer is assigned to the self argument. 

This is a very important argument assignment. Without it, the Python interpreter 
cani work out which object instance to apply the method invocation to. Note 
that the class code is designed to be shared among all of the object instances: 
the methods are shared, the attributes are not. The self argument helps 
identify which object instance 5 s data to work on. 
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custom data objects 


Every wethod'$ first argument is sejf 

In fact, not only does the_init_() method require self as its first 

argument, but so does every other method defined withinyour class. 

Python arranges for the first argument of every method to be the invoking 
(or calling) object instance. Let’s extend the sample class to store a value in a 
object attribute called thing with the value set during initialization. Another 
method, called how_big (), returns the length of thing due to the use of 
the len () BIF: 


The Vrl” tode no* 
assius a suyylied vata 
to a elass attribute 
talled w self thing • 


class Athlete: 


def 




self.thing = value 


def how_big(self): \ 

return(len(self.thing)) 



The l 'h°w bigO" meihod retums the 
le«gth of self.thing". 


No-fce -the use of 
W *l*f W -to idchtify 


When you invoke a class method on an object instance, Python arranges for 
the first argument to be the invoking object instance, which is always assigned 
to each method’s self argument. This fact alone explains why self is 
so important and also why self needs to be the first argument to every object 
methodyou write : 


What you write: 


What Python exeeutes: 
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idle session 


An IDLE Session 

/ 


Let's use IDLE to create some object instances from a new class that you'll define. Start by creating a small class 
called Athlete: 

^- Th* f \»...J . i 

roir 



»> class Athlete 

def _init_(self, a_name, a_dob=None, a_times=[]): 

self.name = a_name 
self.dob = a dob 


‘AM* -thc de-Pault values £ 
vho o-f -the avguments. 


self.times = a times 



TW **"' £la “ 

atbriWtes ^ ^e s *?? lied data 


With the class defined, create two unique object instances which derive their characteristcs from the Athlete 
class: 



»> sarah = Athlete('Sarah Sweeney', '2002-6-17', ['2:58', '2.58', '1.56']) 
»> james = Athlete ('James Jones ') 

»> type (sarah) 

<class '_main_. Athlete'> 

»> type (j ame s) 

<class ' main .Athlete 


Con-fivm that both "savaV» and 
t> -"james” ave athletes. 


• Cvea-te two uni^ue athletes (with 
USI * 9 the de * auli 


Even though sarah and j ames are both athletes and were created by the Athlete class's factory function, 
they are stored at different memory addreses: 


»> sarah 

< main_.Athlete object at 0xl4d23f0> 

»> james 


<_main_.Athlete object at 0xl4cb7d0> 



These » tt. 

diffo- W tte «*« «rt /ffl, 

the memovv addvess U “sarah and james 


Now that sarah and j ames exist as object instances, you can use the familiar dot notation to access the 
attributes associated with each: 

»> sarah.name 
'Sarah Sweeney' 

»> j ames . name 
'James Jones' 

»> sarah.dob 
'2002-6-17' 

»> james.dob 
»> sarah. times 
['2:58', '2.58', '1.56'] 

»> james. times 

[] 


The james” object instance has no value t, 
dob , so nothing appeavs on sev-een. 
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custom data objects 



Here's your code (except for the santize () function, which doesn't 
need to change). With your pencil, write code to define the Athlete 

class. In addition to the_ init _() method, define a new method 

called top3 () that, when invoked, returns the top three times. 


Be sure to adjust the get_coach_data () function to return an 
Athlete object as opposed to a dictionary, and don't forget to amend 
yourprint() statements, too. 


Wri-te your Athlete 
Code heire- 


def get_coach_data(filename): 
try: 


with open(filename) as f: 

data = f.readline() 
templ = data.strip() .split(’, 1 ) 
return({’Name' : templ.pop(0), 

'DOB’ : templ.pop(0), 

'Times': str(sorted(set([sanitize(t) for t in templ]))[0:3])}) 
except IOError as ioerr: 

print('File error: ' + str (ioerr)) 
return(None) 



james = get coach data('james2.txt') 


print(james[’Name'] + M, s fastest times are: 


n 


+ 


This line o-f £ode 
«eeds io change, ioo. 



j ames[’Times’]) 
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class athlete 



Here's your code (except for the santize () function, which doesn't 
need to change). With your pencil, you were to write code to define the 

Athlete class. In addition to the_ init _() method, you were to 

define a new method called top3 () that, when invoked, returns the top 
three times. You were to be sure to adjust the get_coach_data () 
function to return an Athlete object as opposed to a dictionary, and 
you weren't to forget to amend print (), too. 


dlass 



def get_coach_data(filename): 
try: 

with open(filename) as f: 

data = f.readlineO 
templ = data.strip().split(’,’) 


Refove the dictio»av-y creatio* 

t de «P 1 ** Athlete 
object creabo» code iwtead. 


return (<( , 

'ttlllH' , Athlete(-ternp|.J>op(0), ternpl.pop(O), tew>fl) 


1 1 mos 1 • s t g f, n e i^t n n , , cgt '^ i i i i f i ;i r \ i 


ujj.i 1 1) anm ) 


except IOError as ioerr: 

print('File error: ' + str (ioerr)) 

return (None) Use the dot hotatioh 

at y°uv data. 

james = get coach data('james2.txt') 


james.name 

print (j ^mc j + M 's fastest times are: " + j amoo [ ' Timeo - 1 - ] ) 


stv-(ja rh ts .0 ) 


Imo ke the w top l() tt 
method ahd £ohvert its 
^esults to a stv-ing pHov 
to its display oh s£\reeh. 
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custom data objects 



'esT DRive 


With these changes applied to your program, let's ensure you continue to get the same results as 
earlier. Load your code into IDLE and run it. 


* ^ O 


coach4.py - /Users/barryp/HeadFirstPython/chapter6/coach4.py 


lass Athletes 

def _ .init (self, a name, a_dob=None, a times=[])s 

self.name = aname 
self.dob = a dob 
self.times = atimes 

def top3 (self )2 

return( sorted(set ([sanitize(t) for t in self.times]))[0 2 3]) 


def 



get coach data( filename )2 

try 2 

with open(filename) as f 2 
data = f.readline() 
templ = data.strip().split( ’,’ ) 
return(Athlete(templ.pop(0) 
except IOError as ioerr 2 

print(’File error: 1 + str(ioerr)) 
return(None) 


james = get coach data(’ james2.txt ’) 
julie = getcoach data(’ julie2.txt 1 ) 
mikey = get coach data( 'mikey2.txt ') 
sarah = get coach data(' sarah2.txt ') 


The tode. h> the Wr«0* Wfcio* is 
l 0t Sh ° wn he " e ' ^ it is stili par t of this 


templ.pop(0), templ)) 


print( james.name + 
print( julie.name + 
print (mikey.name + 
print( sarah.name + 


ii i 


ii i 


ii i 


ii i 


s fastest times 
s fastest times 
s fastest times 
s fastest times 


are 2 
are 2 
are 2 
are 2 


+ str( james.top3())) 
+ str (julie.top3())) 
+ str (mikey.top3())) 
+ str (sarah.top3())) 


«00 


Python Shell 


»> 

»> 


RESTART 


James Lee p s fastest times are: ['2.01 1 , F 2.16 F , F 2.22 F 


Julie Jones 1 s fastest times aret 
Mikey McManus F s fastest times are: 
Sarah Sweeney F s fastest times are: 
»> I 


2 * 11 F , F 2.2 3 F , 2 . 59 F ]| 

[ 1 2.22 ’ , F 2 . 3 1 F , 2.38 

[ F 2 * 1 8 F , F 2.2 1 p , 2.22 



Cool! T\\ert* *° 
dha^ge heve- 



And to make objects 
do more, I just add more 
methods, right? 


Ln: 30 


Coi: 4 


0 




Yes, that’s correct: more functionality = more methods. 

Simply add methods to encapsulate the new functionality you need within 
your class. There 5 s no limit to how many methods a class can have, so feel 
free to knock yourself out! 

you are here ► 


197 
































no dumb questions 


tfiereicire no 

Dumb QuestiQtis 


O: l’m not sure I see why the top3() method is coded to return 
a three-item list, as opposed to a string? Surely a string would 
make the print() statement in the main program easier to write? 


It would, but it wouldrft be as flexible. By returning a list (albeit 
a small one), the top3 () method lets the calling code decide what 
happens next, as opposed to forcing the caller to work with a string. 
Granted, the current program needs to treat the list like a string, but 
not all programs will want or need to. 


Q.: Why does the class even need the top3() method? Why 
not store the top three times as an attribute within the class and 
create it as part of the objecfs creation? 


Again, better not to, because doing so is less flexible. If you 
compute and store the top three times at object creation, you make it 
harder to extend the list of timing data associated with the object. 


For instance, if you add more timing data after the object is created, 
you’ll need to arrange to recompute the top three (because the new 
times might be fast) and update the attribute. However, when you 
compute the top three times “on the fly” using a call to the top3 () 
method, you always ensure you’re using the most up-to-date data. 

OK, I get that. But, with a little extra work, I could do it 
during object creation, right? 

Well, yes...but we really don’t advise that. By preserving the 
original data in each objecfs attributes, you are supporting the 
extension of the class to support additional requirements in the 
future (whatever they might be). If you process the data as part of 
the object initialization code, the assumptions you make about how 
programmers will use your class might just come back to bite you. 


Q.: But what if l’m the only programmer that’11 ever use a 
custom class that I write? 


Trust us: you’ll thank yourself for coding your class to be as 
flexible as possible when you come to use it for some other purpose 
in a future project. When you are creating a class, you have no idea 
how it will be used by other programmers in their projects. And, if you 
think about, you have no idea how you might use it in the future, too. 


OK, I think Tm convinced. But teli me: how do I go about 
adding more times to my existing Athlete objects? 


To do more, add more methods. With your Athlete class 
created, ifs a breeze to extend it to do more work for you: simply add 
more methods. 


So, if you want to add a single new timing value to your times 
attribute, define a method called add_t ime () to do it for you. 
Additionally, you can add a list of times by defining a method called 
add_times () .Then all you need to do in your code is say 
something like this: 

sarah.add_time(' 1 . 31 ') 
to add a single time to SarafVs timing data, or say this: 

james.add_times([' 1 .2 1 ' , '2.22']) 
to add a bunch of times to James’s data. 


But surely, knowing that times is a list, I could write code 
like this to do the same thing? 

sarah.times.append('1.31’) 
j ames.times.append(['1.21 ', '2.22’]) 


A 


I You could, but that would be a disaster. 

Q.: What?!? Why do you say that? There’s nothing wrong 
with my suggestion, is there? 

Well...it does indeed work. However, the problem with writing 
code like that is that it exposes (and exploits) that fact that the 
timing data is stored in a list within the Athlete class. If you 
later change your class implementation to use (for instance) a string 
instead of a list, you may well break all of the existing code that uses 
your class and that exploits the fact that the timing data is a list. 


By defining your own API with add_time () and add_ 
times (), you leave open the possibility that the way the data 
is stored within your class can change in the future (obviously, only 
if such a change makes sense). It is worth noting that one of the 
reasons for using object orientation is to hide away the details of a 
class’s implementation from the users of that class. Defining your 
own API directly supports this design ideal. Exposing the intemals of 
your class’s implementation and expecting programmers to exploit it 
breaks this fundamental ideal in a very big way. 
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custom data objects 



Let’s add two methods to your class. The first, called add_time (), appends a single 
additional timing value to an athlete’s timing data. The second, add_times (), extends an 
athlete’s timing data with one or more timing values supplied as a list. 

Here’s your current class: add the code to implement these two new methods. 

class Athlete: 

def _init (self, a_name, a_dob=None, a_times=[]): 

self.name = a_name 
self.dob = a_dob 
self.times = a times 


def top3(self): 

return(sorted(set([sanitize(t) for t in self.times]))[0:3]) 


J\dd youv- new 
methods hev-e. 



%J>arpen your pencil 



Don't put down the pencil just yet! Provide a few lines of code to 
test your new functionality: 


you are here ► 
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more methods 



SoLutiort 


Let’s add two methods to your class. The first, called add_time (), appends a single 
additional timing value to an athlete’s timing data. The second, add_times (), extends an 
athlete’s timing data with one of more timing values supplied as a list. 

Here’s your current class: you were to add the code to implement these two new methods. 


class Athlete: 

def _init_(self, a_name, a_dob=None, a_times=[]): 

self.name = a_name 
self.dob = a_dob 
self.times = a times 


def top3 (self) : 

return(sorted(set([sanitize(t) for t in self.times]))[0:3]) 




While stili holding on firmly to your pencil, you were to provide a 
few lines of code to test your new functionality: 


Create a new 

olojctt instante 
-Cor Vera. 



vera =■ Athlete(Vera VV) 
vera.add_tm»e03f) ^ 


Add a single timing value- 


print(verato{30) -This will disflay a list with only one value in it : 131. 

veva.add_ti«.es(f2~22', *I-2J", 'Z-ZZ'1) -^ dd thlree *** ^»9 valu «- 

pvmtveva-fcopSO) -=«=-The tp i iWmg stoves ave vove 1-2-1, 111 a*d 2-22- 
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custom data objects 



Amend your code with the updated version 
of your Athlete class before proceeding 
with this Te st Drive. 



After running your existing program, try out your test code in the IDLE shell to confirm that 
everything is working as expected. 


*O0 


Python Sheli 


RE5TART 


»> = = = = = = = = = = = = = = = ; 

»> 

James Lee p s fastest times are: [ 1 2 * 01 F , P 2*16 P , p 2 

Julie Jones p s fastest times are: ['2.11', p 2*23 p , 

Mikey McMarms p s fastest times are: [ p 2.22 p # p 2.31 p 
Sarah Sweeney ' s fastest times are: [ f 2 * 1S ' # V2.2i p 

>» vera = Athlete( 'Vera Vi' )<:—‘ Cveate a xew athlete- 

>>> vera.add_time( '1.31') ^-/\dd o«e W) value- 

»> prrnt ( vera. top3 ( ) ) 7 J 

[ p 1 * 31 p ] 

»> 

»> vera.add times( [ 1 2,22 1 
»> print ( vera. top3 ( ) ) 

[ p 1 * 21 p , P 1 * 31 p , 1 2 - 2 2 1 ] 

»> I 



M expetlcd- 


Pisplay the top three 'times (there s only eme, so thdts all you see). 
1 1-21 11 f r 2 i 22 p ] ) <;- dd thvee mov-e tim’m<} values. 

P*, s p|ay the top three "times (whidh, no\w, makes a little move sense)- 


0 


M 


Ln: 179 Coi: 4 


Great: it worked. 

You Ve packaged your code with your data and created a custom class 
from which you can create objects that share behaviors. And when extra 
functionality is required, add more methods to implement the required 
functionali ty. 

By encapsulating your athlete code and data within a custom class, you Ve 
created a much more maintainable piece of Software. You will thank 
yourself for doing this when, in six months, you need to amend your code. 

Well done. This is really coming along! 


you are here ► 


201 






















reinventing the wheel 



Emmm...maybe I'm missing something, but 
isn't your Athlete class wastef ul? I mean, 
youve extended it with func+ionali+y thafs 
already in lists, which feels a little like 
reinventing the wheel to me... 


Yes, your Athlete class is much like a list. 

Your Athlete class does indeed behave like a list most 
of the time , and youVe added methods to expose some list 
functionality to the users of your class. But it 5 s true :you are 
reinventing the wheel here. Your add_time () method 
is a thin wrapper around the list append () method and 
your add_times () method is list 5 s extend () method in 
disguise. 

In fact, your Athlete class only differs from Python 5 s list 
due to the inclusion of the name and dob object attributes. 
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custom data objects 



Wouldn't it be dreamy if there were a 
way to extend a built-in class with custom 
attributes? But I know it's iust a fantasy... 


you are here ► 
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inherit class 


Inherit frow Python's built-in list 


Pythoni class lets you create a custom class from scratch/]\i$t like you did with 
your Athlete class. However, class also lets you create a class by inheriting 
from any other existing class, including Pythorfs built-in data structure classes 
that provide you with list, set, and dict. Such classes are referred to as 
subclasses. 


What’s really nice is that when you inherit from an existing class (such as 
list), you are given all of the existing functionality for free. 


As your existing class is really nothing more than a list with added attributes, 
perhaps a better design is to kill off your Athlete class and replace it 
with a class that inherits from the built-in list class? It’s certainly worth 
considering, isn’t it? 



E 


Herein lies the Athlete class. 

'A short, but glittering, 
career. 

Will be missed by all who 
knewandlovedhim. 

Taken in his prime... 




Sorry to hear about your 
Athlete class. But, according 
to my files, youre in line to inherit a 
mountain of functionality from the 
built-in list class. Congratulations, 
youre rich! 


r 1 

Slippery 
ldwye\r--typ c 
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custom data objects 


Tonightfs talk: Inheritance, a.k.a. He looks just like his father. 


Custom Class: Inherited Class: 

Programmers like me because they get to 
control everything in their code.. .and you know 
programmers: they love to code. 

Yes, they do. But sometimes writing everything from 
scratch is not the best design decision. 

Design! Phooey! Real programmers eat, sleep, 
dream, snore, and exhale code. All that design talk 
is for people who cardt code! 

Is it really? So, you 5 re saying it 5 s much better to do 
everything from scratch and repeat the work of 
others, because your way is the best way. Are you 
serious?!? 

No, no, no: you’re not listening. It’s all done with 
control. When you build everything from the 
ground up, you’re in control, as itfs all your code. 

And you 5 re happy to reinvent the wheel, even 
though someone else solved that problem eons ago? 

Of course, especially when there are custom 
requirements to be taken into consideration. In that 
case, a brand-spanking new custom class is the only 
way to go. 

Not if you can extend someone else 5 s class to handle 
your custom requirements. That way, you get the 
best of both worlds: inheritied functionality (so 
you 5 re not reinventing the wheel) together with the 
custom bits. It’s a win-win situation. 

Yeah, right.. .it’s a win-win for you, not me. 

But it’s not about us: it’s to do with making the life 
of the programmer easier, even the ones that live to 
code, right? 

I guess so, although fm stili a fan of custom code... 


Fireside Chats 
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idle session 


An IDLE Session 

/ 


Let's see what's involved in inheriting from PythorVs built-in list class. Working in IDLE's shell, start by creating a 
custom list derived from the built-in list class that also has an attribute called name: 


»> class NamedList (list) 


Pvovide the hame of the class that this 
^lass dcmivcs -(Vor*. 


def _init_(self, a_narae) : 

list._init_ ([]) 

self.name = a name ftVtialize the dcv-Wcd *fv*ow» tlass, and then 




assty -bKc av-^umcnt to the attribute- 


With your NamedList class defined, use it to create an object instance, check the object's type (using the 
type() BIF), and see what it provides (using the dir () BIF): 


Create a «eu/ "NamedList" object inst aue- 


»> johnny = NamedList("John Paul Jones") 

»> type (johnny) 

<class '_main_. NamedList ’> \s d W KamcdList W - 

»> di r (j ohnny) 

[’ add ', ’ class ', ' contains ', ’ delattr ', ’ delitem ', ’ dict ', ’ doc 

_eq ’ , ’ format ’ , ’ ge ’ , ’ getattribute ’ , ’ getitem ’ , ’ gt ’ , ’ hash ’ , 


i add 


f f 


imul 


f f 


init 


f f 


iter 


f f 


le 


i i 


len 


f f 


lt 


f f 


module 


mul_’ , ’_ne_’ , ’_new_’ , ’_reduce_’ , ’_reduce_ex_’ , ’_repr_’ , ’_reversed_’ , 

_rmul_’ , ’_setattr_’ , ’_setitem_’ , ’_sizeof_’ , ’_str_’ , ’_subclasshook_’ , 

_weakref_', 'append', 'count', 'extend', 'index', 'insert', 'name', 'pop', 'remove', 

' 'sort'] 


reverse 



T' 

>™ y * u, d. a u «», * «II * ^ Ma i„ tt, 

Use some of the functionality supplied by the list class to add to the data stored in j ohnny: 


»> johnny.append("Bass Player") 

»> johnny.extend(['Composer', "Arranger", "Musician"]) 
»> j ohnny 

['Bass Player', 'Composer', 'Arranger', 'Musician'] 

»> j ohnny. name 
1 John Paul Jones' <e— 


Add data to the "NamedList” the 
methods f-rovided by the list bwilt m. 



Access the list data, as well as 
the attribute data. 


Because j ohnny is a list, it's quite OK to do list-type things to it: 

»> ,« i. > , s fkt ^ ^ fc, 

print(johnny.name + " is a " + attr + ".") ^ J . ' ) i - |- 1C 1 

J use it wherever you d use a hst- 

John Paul Jones is a Bass Player. 

John Paul Jones is a Composer. 

John Paul Jones is a Arranger. 

John Paul Jones is a Musician. 


t 
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Co^irmation: Oohns a busy boy. © 












custom data objects 



Here is the code for the now defunct Athlete class. In the space provided below, rewrite this 
class to inherit from the built-in list class. Call your new class AthleteList. Provide a 
few lines of code to exercise your new class, too: 

class Athlete: 

def _init_(self, a_name, a_dob=None, a_times=[]): 

self.name = a_name 
self.dob = a_dob 
self.times = a times 


def top3(self): 

return(sorted(set([sanitize (t) for t in self.times]))[0:3]) 


def add_time(self, time_value): 
self.times.append(time value) 


Wrrte youv 

tlass toAc 



def add_times(self, list_of_times): 
self.times.extend(list of times) 
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new athletelist 



ExeRciSe 

§ 0 (.|#tl 0 H 


Here is the code for the now defunct Athlete class. In the space provided below, you were 
to rewrite this class to inherit from the built-in list class. You were to call your new class 
AthleteList, as well as provide a few lines of code to exercise your new class: 

class Athlete: 

def _init_(self, a_name, a_dob=None, a_times=[]): 

self.name = a_name 
self.dob = a dob 



def top3 (self) : 

return(sorted(set([sanitize(t) for t in self 


s])) [0:3]) 


value): 





def ad 


Tbe dlass 
bas dbanged- “ 



These 
•^e-thods 
freni needed 
ahymoirc. 


f, list of times): 

s) 

Inbevrt -Pvow» "tbe built-m list ^ ss 


dlass ^tblctcListOis*t) : 

de-P_mi-t_(sel-P, a_ name, a__dob— None, a_jtiimes=*[J): 

Wo-thing hcw here... C l ,s t- _ init _ (C3) 

this Code iV ve —■ \ a d ame . 

s.^.lav ifi.thc....../. - . 

War*edLis-t” ihi-t eode. / sel-Pdob =■ a__dob 

sel-Pe%tend(a_jtimes) 

dc-f top3(sel-P) : 


Use the new 
elasss naime 


returnfsorted^set^CsanitizXt) -for t ini 




The data 
i-fesel-P is -the 
Wmg da-ta, 

So -the w tin»es w 
attribute is 
gone. 


vera — AthleteList^Vera \/i0 


Kow that Wve ^ vo-a affe»d( 131 ) .( This tode does a ^ood jolo o£ 

mhevitmo, *' r0 "‘ ^ pvmt(vev-a top^O) / e*evtismo> yowv «e* «-tass. 

f ttl ’ ^ -i-ir;'iii')) 

methods to^et . P vmt(veva.topi6). 


your 


work on- 
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custom data objects 



In your code, replace your Athlete 
class code with your new AthleteList 
class code, and don’t forget to change 
get_coach_data () to return an 
AthleteList object instance as 
opposed to an Athlete object instance. 


tJi©rei£tre no 

Dumb Questipns 


Sorry...but not three minutes ago you were telling me not 
to expose the inner workings of my class to its users, because 
that was fundamentally a bad idea. Now you’re doing the exact 
opposite! What gives? 

Well spotted. In this particular case, its OK to expose the fact 
that the class is built on top of list. This is due to the fact that the 
class is deliberately called AthleteList to distinguish it from 
the more generic Athlete class. When programmers see the 
word “list” in a class name, they are likely to expect the class to work 
like a list and then some. This is the case with AthleteList. 



And I can inherit from any of the built-in types? 



Yes. 



Can I inherit from my own custom classes? 


Of course, that’s the whole idea. You create a generic class 
that can then be “subclassed” to provide more specific, targeted 
functionality. 



Can I put my class in a module file? 


Yes, that’s a really good idea, because it lets you share your 
class with many of your own programs and with other programmers 
For instance, if you save your AthleteList class to a file 
called athletelist. py, you can import the into your code 
using this line of code: 


from athletelist import AthleteList 


O: What about inheriting from more than one class...does 
Python support multiple interitance? 


Yes, but it’s kind of scary. Refer to a good Python reference text 
for ali the gory details. 


then use the class as if it was defined in your current program. And, 
of course, if you create a really useful class, pop it into its own 
module and upload it to PyPI for the whole world to share. 
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te st drive 



One last run of your program should confirm that it's working to specification now. Give it a go in 
IDLE to confirm. 


coach6.py - /Users/barryp/HeadFirstPython/chapter6/coach6.py 

def sanitize( time string): 
if ’ in timestring: 

splitter = 1 - ’ 
elif 1 : 1 in timestring: 

splitter = ' s ' 
else : 

return (time string) 

(mins, secs) = timestring.split(splitter) 
return(mins + + secs) 


class AthleteList ( list )s 


def _ init (self, aname, a_dob=None, a_times=[]): 

list._ init_ ([]) 

self.name = a name 
self.dob = adob 
self.extend(atimes) 

def top3 (self): 

return ( sorted(set( [sanitize(t) for t in self)))[0s3)) 

def get coach data( filename)s 

try s 

with open (filename) as fs 
data = f.readline() 
templ = data.strip().split(',') 

return (AthleteList(templ.pop(0), templ.pop(0), templ)) 
except IOError as ioerr: 

print(’File error: ’ + str(ioerr)) 
return (None) 


james = getcoach data( 1 james2.txt' ) 
julie = getcoach data(' julie2.txt' ) 
mikey = getcoach data(’ mikey2.txt' ) 
sarah = getcoach data(’ sarah2.txt ') 


print( james.name + 
print( julie.name + 
print( mikey.name + 
print (sarah.name + 


's fastest times are: 
' s fastest times are: 
' s fastest times are: 
' s fastest times are: 


+ str( james.top3())) 
+ str( julie.top3())) 
+ str (mikey.top3())) 
+ str (sarah.top3())) 



ftOO 


Python She I [ 


>» S « = = = ^ = S : = = S : = S : = ^ = ES = = ME = = ^ = M = = = = RESTART = = = = = = ^ = = = = = = ^ 
»> 

James Lee r s fastest times are: [ F 2.01 F , P 2.16 P # '2.22 1 ] 

Julie Jones 1 s fastest times are: ( p 2.11 F f F 2.23 F , F 2.59 F ] 

Mikey McManus r s fastest times are: [ F 2*22 F , F 2.3l F , '2.38'] 
Sarah Sweeney r s fastest times are: ( F 2.18 F , F 2*21 F , 1 2.22 r ] 
»> I 


Ln: 306 Coi 4 


A 
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custom data objects 


Coach Kelly is impressed 



By basing your class on built-in functionality you Ve leveraged the power of 
Python 5 s data structures while providing the custom solution your application 
needs. 

You Ve engineered a much more maintainable solution to Coach Kelly 5 s data 
Processing needs. 

Good job! 
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python toolbox 



Your Pythow Toolbox 

You’ve got Chapter 6 under your 
belt and you’ve added some key 
Python techiques to your toolbox. 


assole data r 

,'Yc-i 

**"? ^ ^ -x ** 

’ yM "' *'V > ) 

sb™ t ' yrt 

•»*** 


^Io*-e Pytho* Lingo 

0 Se ^ “ a >»ei}nod a^gumen-t 
that always ire-Pc« to the ducent 
objed-t iw-tande. 



BULLET POINTS 


■ Create a empty dictionary using the 
dict () factory function or using {}. 


■ To access the value associated with 
the key Narae in a dictionary called 
person, use the familiar square bracket 
notation: person [ 1 Name 1 ] . 


■ Like list and set, a Python’s dictionary 
dynamically grows as new data is added 
to the data structure. 


■ Populate a dictionary as you go: 

new d = { } or new d = dict() 

and then 

d['Name'] = 'Eric Idle' 

or do the same thing all in the one go: 

new d = {'Name': 'Eric 
Idle'} 

■ The class keyword lets you detine a 
class. 

■ Class methods (your code) are defined in 
much the same way as functions, that is, 
with the def keyword. 

■ Class attributes (your data) are just like 
variables that exist within object instances. 

■ The_ init _() method can be 

defined within a class to initialize object 
instances. 

■ Every method defined in a class must 
provide self as its first argument. 

■ Every attribute in a class must be prefixed 
with self. in order to associate it data 
with its instance. 

■ Classes can be built from scratch or can 
inherit from Python’s built-in classes or 
from other custom classes. 
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Classes can be put into a Python module 
and uploaded to PyPI. 










7 Web deVelopment 




Putting it all together 




Sooner or later, you’ll want to share your app with lots of people. 

You have many options for doing this. Pop your code on PyPI, send out lots of emails, put 
your code on a CD or USB, or simply install your app manually on the computers of those 
people who need it. Sounds like a lot of work...not to mention boring. Also, what happens 
when you produce the next best version of your code? What happens then? How do you 
manage the update? Let’s face it: it’s such a pain that you’11 think up really Creative excuses 
not to. Luckily, you don’t have to do any of this: just create a webapp instead. And, as this 
chapter demonstrates, using Python for web development is a breeze. 


this is a new chapter 
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caring is sharing 


lt'$ good to sharc 


o 


The coach showed us your program 
running on his laptop...any chance me and my 
friends could also get access to our list of 
times? I'd love to show them to my dad... 



CoacM 

\jO\AY\<$ 


YouYe a victiw of yoor oww success. 

The new requests come flooding in right after Coach Kelly starts showing 
off your latest program. It appears that everyone wants access to the coach’s 
data! 

The thing is: what’s the “best way” to do this? 
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web development 


You caw put your program ow thc Web 



A "webapp" is what you wawt 

If you develop your program as a Web-based applicatiori (or webapp , for short), 
your program is: 

• Available to everyone who can get to your website 

• In one place on your web server 

• Easy to upate as new functionality is needed 

But...how do webapps actually work? 
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anatomy of a web request 



Vebapps XJjp C]<*se - 

No matter what you do on the Web, it 5 s all about requests and responses. A web request is sent 
from a web browser to a web server as the resuit of some user interaction. On the web server, a 
web response (or reply ) is formulated and sent back to the web browser. The entire process can 
be summarized in five steps. 


Step 1: Your user enters 
a web address, selects 
a hyperlink, or clicks a 
button in her chosen 
web browser. 



Step 2: The web 
browser converts 
the user’s action 
into a web request 
and sends itto a 
server over the 
Internet. 


The Internet 


Weve 



Pecidiwg what to do wext 


Step 3: The web server 
receives the web request 
and has to decide what 
to do next. 


Web 

Server 



One of two things happen at this point. If the web request 
is for static content —such as an HTML file, image, or 
anything else stored on the web server 5 s hard disk—the web 
server locates the resource and returns it to the web browser as 

a web response. 

If the request is for dynamic content —that is, content that 
must be generated —the web server runs a program to produce 

the web response. 
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Step 4: The web server 
processes the web 
request, creating a web 
response, which is sent 
back over the Internet 
to the waiting web 
browser. 




Web 

Server 



The (potewtially) wawy substeps of step 4 

In practice, step 4 can involve multiple substeps , depending 
on what the web server has to do to produce the response. 
Obviously, if all the server has to do is locate static content 
and sent it back to the browser, the substeps aren’t too 
taxing, because it’s alljust file I/O. 

However, when dynamic content must be generated , the sub¬ 
steps involve the web server locating the program to exeeute, 
exeeuting the located program, and then capturing the output 
from the program as the web response.. .which is then sent 
back to the waiting web browser. 

This dynamic content generation process has been 
standardized since the early days of the Web and is known 

as the Common Gateway Interface (CGI). Programs 
that conform to the Standard are often referred to as CGI 
Scripts. 





The Internet 



Step 5: The web 
browser receives the 
web response and 
displays it on your 
user’s screen. 
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webapp requirements 


What docs your webapp weed to do? 

Let 5 s take a moment to consider what you want your webapp to look like 
and how it should behave on your user’s web browser. You can then use this 
information to help you specify what your webapp needs to do. 
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r Sharpen your pencil 


web development 


There's nothing like grabbing your pencil and a few blank paper 
napkins to quickly sketch a simple web design. You probably 
need three web pages: a "welcome" page, a "select an athlete" 
page, and a "display times" page. Go ahead and draw out a rough 
design on the napkins on this page, and don't forget to draw any 
linkages between the pages (where it makes sense). 



I 
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back-of-the-napkin sketch 



There's nothing like grabbing your pencil and a few blank paper 
napkins to quickly sketch a simple web design. You probably 
need three web pages: a "welcome" page, a "select an athlete" 




220 Chapter 7 
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Pesigw your webapp with MVC 

Now that you have an idea of the pages your webapp needs to provide, your 
next question should be: whafs the best way to build this thing? 

Ask 10 web developers that question and you’11 get 10 different answers; the 
answer often depends on whom you ask. 

Despite this, the general consensus is that great webapps conform to the 
Model-View-Controller pattern, which helps you segment your webapp’s code 
into easily manageable functional chunks (or components ): 




The Model 

The code to store (and sometimes process) your webapp’s data 



The View 

The code to format and display your webapp’s user interface(s) 



The Controller 

The code to glue your webapp together and provide its business logic 


By following the MVC pattern, you build your webapp in such as way as to 
enable your webapp to grow as new requirements dictate. You also open up the 
possibility of splitting the workload among a number of people, one for each 
component. 

Let’s build each of the MVC components for your webapp. 
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build a model 


Model your data 

Your web server needs to store a single copy of your data, which in this case is 
Goach Kelly 5 s timing values (which start out in his text hies). 

When your webapp starts, the data in the text hies needs to be converted 
to AthleteList object instances, stored within a dictionary (indexed by 
athlete name), and then saved as a pickle file. Let’s put this functionality in a 
new function called put_to_store () . 

While your webapp runs, the data in the pickle needs to be available to your 
webapp as a dictionary. Let 5 s put this functionality in another new function 
called get f rom store (). 



When your webapp starts: 


While your webapp runs: 





The single piekle wi-th 
all o( -the daia sWed 
a dictionary 




W the "yt_W_=W' u» 
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Here is the outline for a new module called athletemodel .py, which provides the 
functionality described on the previous page. Some of the code is already provided for you. Your 
job is to provide the rest of the code to the put_to_store () and get_f rom_store () 
functions. Don’t forget to protect any file l/O calls. 

import pickle 

from athletelist import AthleteList 


def get_coach_data(filename): 

# Not shown here as it has not changed since the last chapter. 


^c>u nccd todt m 
Vievc “to populate 
the dii-Wavy 
the data horr. the 
•files- 


def put_to_store(files 
ali athletes = {} 


list): 




T^ s k^tioir, 1S called with 
list of f ileha*»es as iis sole 
av-gumcb-fc. 



a 


At\d do *-1 -fovgc-t 
to savc -the 
di£-fciohary -to a ' 
(a*d dhedk 

™r -Pile 1/0 evvovs). 


return(all_athletes) 

def get_from_store(): 
ali athletes = {} 


£jet the dittior»avy 
-fv-om tbe -f ilC) so - 
tbat 'A tam be 

vetuftted “to tbc 
dallcv-. 


$otb 'Cun£*l*oir\s 
*eed -to v-etuvn 
a ditWavy of 
AthleteLists- 


return(ali athletes) 
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model module 



ExeRciSe 

§ 0 (.|#tl 0 H 


Here is the outline for a new module called athletemodel .py, which provides the 
functionality described on the previous page. Some of the code is already provided for you. Your 
job was to provide the rest of the code to the put_to_store () and get_f rom_store () 
functions. You were not to forget to protect any file l/O calls. 

import pickle 

from athletelist import AthleteList 
def get_coach_data(filename): 

# Not shown here as it has not changed since the last chapter. 


def put_to_store(files_list): 
all_athletes = {} 

file» W* Vb for eadb -file m -files lisfc 

m-fco a* AthleteUst .- ... tat h athlete's r>a»«e is 

ob\eA instante, and --> «h - 3et_doadh_data(eadh_t.le; used as the "k e y" in the 

add the athletes data . ..' V r- Vi. . i . V. . i . didtionary. The \alue" is 

Z the ditWry all_athletes tath.nar. e3 - ath_ <—-the AthleteList obiedt 

tv-y- 


Savc i' wi-fcb opejr\Ca'thle*tes.piekle , ; 'v/bO as a*tbf: 

pidkle.dump(all__a*tble*tes ; a*tbf) 


di^tiOhairy of 
Athle-fceLis-ts 
■to a piekle. 



e%dep*t lOBrror as ioe\r\r-* 

prirrt( l Pilc erv-or ( pu*t__a y\ d__sfo\re) : * + s*br(ioe\r\r)) 

return(all_athletes) 

def get_from_store(): 
ali athletes = {} 


/W don-t formet 

a -tv-y/e^cp-t -to 

pv-ofetf youv- f ile 

|/0 eode. 


Sim^ply v-ead fbe 
cyvtiv-e piekle mfo 
-the diebio^avy. 
VVVia-b tould be 


tv-y: 


y/i*tb opehOa*tble*tes.pidkle ; , Vb ; ) as a*thf; 
all__a*thletes =■ piekle.loadta-thf) 


easiev ( 


e%dep*t lOBrror as ioev-v--’ 

priivt( l pile e\rv-ov- ( 5 ef__fror*__s-to\re): * + s*br(ioev~\r)) 



^dm-do* f 
fov^ef youv- 
-tv-y/e*£cpt 


return(ali athletes) 
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V 


An IDLE Session 


Let's test your code to ensure that it is working to specificatiori. Type your code into an IDLE edit window and save 
your code into a folder that also includes the coach's text files. Press F5 to import your code to the IDLE shell, and 
then use the dir () command to confirm that the import has been successful: 

»> dir () 

[' AthleteList 1 , '_builtins_' , '_doc_', '_name_' , '_package_', ' get_coach_data' , 

' get_from_store', 'pickle', 'put_to_store'] 

Create a list of files to work with, and then call the put_to_store () function to take the data in the list of files 
and turn them into a dictionary stored in a pickle: 


»> the_files = ['sarah.txt', 'james.txt', 'mikey.txt', 'julie.txt'] 
»> data = put_to_store (the_files) 

»> data 


IWs ali of the 
AthleteLists. 

C 


{'James Lee': ['2-34', '3:21', '2.34', '2.45', '3.01', '2:01', '2:01', '3:10', ' 2 - 22 ', '2- 

01', '2.01', '2:16'], 'Sarah Sweeney': ['2:58', '2.58', '2:39', '2-25', '2-55', '2:54', '2.18', 
'2:55', '2:55', '2:22', '2-21', '2.22'], 'Julie Jones': ['2.59', '2.11', '2:11', '2:23', '3- 

10', '2-23', '3:10', '3.21', '3-21', '3.01', '3.02', '2:59'], 'Mikey McManus': ['2:22', '3.01', 
'3:01', '3.02', '3:02', '3.02', '3:22', '2.49', '2:38', '2:40', '2.22', '2-31']} 


At this point, the athletes . pickle file should appear in the same folder as your code and text files. Recall 
that this file is a binory file, so trying to view it in IDLE or in your editor is not going to make much sense. To access 
the data, use the dictionary returned by the put_to_store () or get_f rom_store () functions. 

Use the existing data in the data dictionary to display each athlete's name and date of birth: 

»> for each_athlete in data: 

print(data[each_athlete].name + ' ' + data[each_athlete].dob) 


James Lee 2002-3-14 
Sarah Sweeney 2002-6-17 
Julie Jones 2002-8-17 
Mikey McManus 2002-2-24 





Bv at£ess''»5 -t^e "«a»>e a »>d dot> , c 

attvibutes, 70« ta» yt at the «st 
the AthleteList data- 


Use the get_f rom_store () function to load the pickled data into another dictionary, then confirm that the 
results are as expected by repeating the code to display each athlete's name and date of birth: 

»> data_copy = get_from_store () 

»> for each_athlete in data_copy: 

print(data copy[each athlete].name + ' ' + data copy[each athlete].dob) 


James Lee 2002-3-14 
Sarah Sweeney 2002-6-17 
Julie Jones 2002-8-17 
Mikey McManus 2002-2-24 



The data i* the vetuirhed dietioharv 

I* * e *aetly the sar» e as 1 

that p«duded by put_to_stoveO. 


you are here ► 


225 






interface view 


View your interface 

With your model code written and working, it 5 s time to look at your view code, 
which creates your webapp’s user interface (UI). 

On the Web, Uls are created with HTML, the Web’s markup technology. If 
you are new to HTML, it is worth taking some time to become familiar with 
this critical web development technology There’s lots of material on the Web 
and more than a few good books out there. 




tNoic -fv-onn /Wav-ke-tihg: 
This is -the book -tha-t we 
'reiommemd -fov <p»ickly 
ge-ttihg wp -to speed wi-fch 
nT/VJi—not that y/eVe 
biased ov anything. © J. 


Hey, we hear you are getting in+o 
web development? We have a small 
module that we put together that might help 
you generate HTML It's a little rough, but it 
works. Youre more than welcome to use it for 
your projects, if you like. 


(Mosi 0 () the Head Fivs-t 
Code Review Te^r* 



YATE: Yet Ahother Template Engine 



Your friends over at the Head First Code Review Team heard you’re planning 
to write some code to generate HTML for your webapp 5 s UI. TheyVe sent 
over some code that they swear will make your life easier. It’s a small library 
of HTML-generating helper functions called yate. The code was produced 
quickly and was originally designed to be “throw away,” so the team has 
provided it as is. It’s somewhat raw, but it should be OK. 
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from string import Template 

def start_response(resp="text/html"): 

return('Content-type: ' + resp + '\n\n') 

def include_header(the_title): 

with open('templates/header.html') as headf: 



head text = headf.read() 


header = Template(head_text) 

return(header.substitute(title=the title)) 


def include_footer(the_links): 

with open('templates/footer.html') as footf: 


foot_text = footf. read() \J 

link_string = '' 
for key in the_links: 

link_string += '<a href="' + the_links[key] + + key + '</a>&nbsp;&nbsp;&nbsp;&nbsp;' 

footer = Template(foot_text) 

return(footer.substitute(links=link_string)) 

def start_form(the_url, form_type="POST"): 

return('<form action="' + the_url + '" method="' + form_type + 

def end_form(submit_msg="Submit"): 

return ( '<p></pxinput type=submit value="' + submit_msg + 

def radio_button(rb_name, rb_value): 

return ('<input type="radio" name="' + rb_name + 

'" value="' + rb_value + '"> ' + rb_value + '<br />') 

def u_list(items): 

u_string = '<ul>' 
for item in items: 

u_string += '<li>' + item + '</li>' 
u_string += '</ul>' 
return (u string) 

def header(header_text, header_level=2): 

return('<h' + str(header_level) + '>' + header_text + 

'</h' + str(header_level) + '>') 

def para(para_text): 

return('<p>' + para_text + '</p>') 
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template engine code 




Let’s get to know the yate code before proceeding with the rest of this chapter. For each chunk 
of code presented, provide a written descriptiori of what you think it does in the spaces provided: 


from string import Template 



Mri-te your 

CXpIdholtiOhS ih 

■the spaces. 



def start response(resp="text/html"): 

retura('Content-type: ' + resp + '\n\n') 

^ This 'PuKdtioK iakes a sm^le (op-tio^al) s-brir^ as i*ts av-0}ur*errt a*d uses i*t *to 
dv-ea*te a C£jl u Cor\'teh*t-*type :W Ime, wi*th as *the de-fault 


teeh dohe -fov- you. 


def include header(the title): 

with open('templates/header.html') as headf: 

head_text = headf.read() 
header = Template(head text) 
return(header.substitute(title=the title)) 


def include footer(the_links): 

with open('templates/footer.html') as footf: 

foot_text = footf.read() 
link string = '' 
for key in the links: 

link_string += '<a href= M ' + the_links[key] + '+ key + 

'</a>&nbsp;&nbsp;&nbsp;&nbsp;' 

footer = Template(foot_text) 

return (footer.substitute(links=link_string)) 
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def start form(the uri, form type= M POST"): 

return('<form action="' + the uri + '" method="' + form type + ' ">') 


def end form(submit msg= M Submit M ): 

return('<p></p><input type=submit value="' + submit msg + '"></form>') 


def radio button(rb name, rb value): 

return('<input type= M radio M name="' + rb name + 

value="' + rb value + ,M > ' + rb value + '<br />') 


def u_list(items): 

u string = '<ul>' 
for item in items: 

u string += '<li>' + item + '</li>' 
u string += '</ul>' 
return(u string) 


def header(header text, header level=2): 

return ('<h' + str(header level) + '>' + header text + 
'</h' + str(header level) + '>') 


def para(para_text): 

return('<p>' + para text + 1 </p> T ) 
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template engine described 



SoLotioH 

Let's get to know the yate code before proceeding with the rest of this chapter. For each chunk 
of code presented, you were to provide a written descriptiori of what you think it does: 

from string import Template 

(mport the u 7emplate W dlass -from *tbe standard library^s "string 
L, i i * module* This allov/s -for simple strinft -substitution templates. 

Nde thedefault ^. ■. . . .. 1 . 

^ def start response(resp="text/html"): 

return('Content-type: ' + resp + '\n\n') 

This -fundtion takes a sih^le (optional) string as its ar<)ument and uses it to 
dreate a C^jl w Conte nt—type :u line, with w te*t/html w as the de^fault* 

Opev> the def include_header (the_title) : 

template -file with open('templates/header.html') as headf: 

(y,With is _^ head_text = headf.read() 

V-ead it in» and^ header = Template (head text) 

substitute in ^ ^ return (header. substitute (title=the title) ) 

pvovided title '// * " ^ 

v ^ This -fundtion takes a sinjjle string as its av-^ument and uses at the title -for 

the start o-f a HTML pa^e* The pay itsel-f is stored within a separate -file 
in w templates/header*htmr , ; and the title is substituted in as needed* 

Open the template 
tile fwhidh is 

11 *T*ii a i \ def rnclude footer (the lrnks) : 

HTML), read ii i„, 

- j i /./ / , with open('templates/footer.html') as footf: 

and substitute m the 

pv-ovided didtionavy foot_text = footf.reado 

of HTML lmks j n link_string = ' ' 

the links' for key in the links: t*i • i i . . . 

' ~ 1 h,s looks a little 

link_string += '<a href= M ' + the_links[key] + + key + weivd but it* 

// ' </a>&nbsp;&nbsp;&nbsp;&nbsp; ' lLtmvm ' 

/ pr_-- n IML hadk fov- 

/ footer = Template (foot text) ‘ C . 

Tum the / “ spaees 

, 0' return (footer. substitute (links=link string)) ^,1. * _ /. • 

didtionary o\ -- y ,hto a s tring. 

links in*to a string, Simi lar to the U indlude__header” -fundtion, this one uses its sin^le string as 
^^t^^d^into av "5 umC ^ *^° tvcakc the end o£ a HTML page* The pa$e itsel-f is stored 

the template* within a separate -file in w templates/'footer*htm| w , and the ar^ument is used 

to dynamieally dreate a set o-f HTML link tags. Based on how they are used, 
it looks like the ar^ument needs to be a didtionary* 
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TV.s is -typidally ei-fchev 

'POST" ov "$£T". 

def start form(the uri, form type= M POST M ): 

return('<form action= M ' + the uri + method="' + form type + 

Tbis -Pu*dtio* retur*s tbe HTML -for tbe start o-f a -form a*d lets tbe daller 
spedi-Py tbe URL to se*d tbe -formas data to, as y/cll as tbe metbod to use- 



def end form(submit msg= M Submit M ): 

return (' <p></p><input type=submit value=" ' + submit msg + '"></form>') 

Tbis -Pu*dtio* retur*s tbe HTML markup, v/bidb termi*ates tbe -form wbile 
allov/ih^ tbe daller to dustomize tbe te%t o-f tbe -fornr/s w submit” butto*. 


def radio button(rb name, rb value): 

return ('<input type= M radio M name="' + rb name + 

'" value="' + rb value + '"> ' + rb value + '<br />') 

£jjive* a radio-butto* *ame a*d value, dreate a HTML radio butto* (wbidb is 
typidally 'mdluded v/itbi* a HTML -form). Note : botb ar<)ume*ts are re^uired- 


J\ simple w *for w 
loop does tbe 
tridk- 


def u_list(items): 

u string = '<ul>' 
for item in items: 

u string += '<li>' 
u string += '</ul>' 
return(u string) 



+ item + 


'</li>' 


6\\vtY\ a Ii st <y f ite ms, tbis -Pu*dtio* tur*s tbe list mto a HTA1L u**umbered 
st- A simple u -Por” loop does all tbe y/ork, addi*$ a L| to tbe UL eleme*t 


y/itb eadb iteratio*. 


def header(header text, header level=2): 

return('<h' + str(header_level) + '>' + header_text + 

1 </h 1 + str(header level) + '>') 

Create a*d retur* a HTML beader ta$ Otl, HZ, HZ, a*d so o*) v/itb level 2. 
as tbe de-Pault- Tbe w beader__te%t” argumerrt is re«\uired- 


def para(para_text): 


return ( '<p>' + para text + 1 </p> 1 ) 

£*dlose a paragrapb o-f te%t (a stri*^) i* HTML parajrapb tay. /\lmos-L *ot 
v/ortb tbe e£Port, is it? 
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no dumb questions 


tlierejcire no 

Dumb Qtiesti9ns 


q: Where are the HTML templates used in the include. 
headerQ and include_footer() functions? 


They are included with the yate module’s download. Go 
ahead and grab them from the Head First Python support website, 
and put them into a folder of your choice. 



And you did this because you are using MVC? 


Partly, yes. The reason the MVC pattern is being followed is to 
ensure that the model code is separate from the view code, which 
are both separate from the controller code. No matter the size of the 
project, following MVC can make your life easier. 


Q/ Why do I need yate at ali? Why not include the HTML that I 
need right in the code and generate it with print() as needed? 


You could, but it’s not as flexible as the approach shown 
here. And (speaking from bitter experience) using a collection of 
print () statements to generate HTML works, but it turns your 
code into an unholymess. 



But surely MVC is overkill for something this small? 


We don’t think so, because you can bet that your webapp wili 
grow, and when you need to add more features, the MVC “separation 
of duties” really shines. 


i— tf / en IDLE Session 


Let's get to know the yate module even more. With the code downloaded and tucked away in an easy-to- 
find folder, load the module into IDLE and press F5 to take it for a spin. Let's start by testing the start_ 
response () function.The CGI Standard States that every web response must start with a header line that 
indictes the type of the data included in the request, which start response () lets you control: 


The dc-Paul-t C6\ \respohse headev-, 
plus vaHatiohs oh a -theme- 


»> s tart_response () 

'Content-type: text/html\n\n 1 
»> start_response ("text/plain") 

'Content-type: text/plain\n\n' «e 
»> start_response ("application/json") 

'Content-type: application/json\n\n' 

The include_header () function generates the start of a web page and let's you customizee its title: 

»> include_header ("Welcome to my home on the web! ") 

'<html>\n<head>\n<title>Welcome to my home on the web!</title>\n<link type="text/css" 
rel="stylesheet" href="/coach.ess" />\n</head>\n<body>\n<hl>Welcome to my home on the web!</ 
hl>\n' 




Tku ali k* a HH.« -* 

b ka ,^4 k Y t ll HTML. Nota 

wW.» ot a w. b, a C£S «a (~*a «. th» ■" a ^ 


232 Chapter 7 









web development 


The include_f ooter () function produces HTML that terminates a web page, providing links (if provided as a 
dictionary). An empty dictionary switches off the inclusion of the linking HTML: 


»> include_footer({'Home': '/index.html', 'Select': '/cgi-bin/select.py'}) 

'<p>\n<a href="/index.html">Home</a>&nbsp;&nbsp;&nbsp;&nbsp;<a href= M /cgi-bin/select. 
py M >Select</a>&nbsp;&nbsp;&nbsp;&nbsp;\n</p>\n</body>\n</html>\n' 

»> include_footer ({}) 

'<p>\n\n</p>\n</body>\n</html>\n' N. liwks mtludcd» d^d 

^ vii'thou'b- 

The start_f orm ( ) and end_f orm ( ) functions bookend a HTML form, with the parameter (if supplied) 
adjusting the contents of the generated HTML: 



»> start_form ("/cgi-bin/process-athlete.py") 

'<form action="/cgi-bin/process-athlete.py" method="POST">' 
»> end_form() 

' <pX/pXinput type=submit value="Submit"X/form>' 


The avgunneht allows you to spedi-Py 
the »a«>e o+ -the proqv-ar» on -the 
fc se , d the ^ di £ l. 


»> end_form ("Click to Confirm Your Order") 

'<pX/pXinput type=submit value="Click to Confirm Your Order"X/form>' 


HTML radio buttons are easy to create with the radio_button () function: 
»> for fab in [' John' , ' Paul' , ' George' , ' Ringo' ] : 


radio button(fab, fab) 


Whith onC is your Parorite? 
Seledt Pror» the Itet of radio 


'<input type="radio" 
'<input type="radio" 
'<input type="radio" 
'<input type="radio" 


name="John" value="John"> John<br />' 
name="Paul" value="Paul"> PauKbr />' 
name="George" value="George"> George<br 
name="Ringo" value="Ringo"> Ringo<br /> 


^ buttons- 

V 


Unordered list are a breeze with the u_list () function: 

u_list(['Life of Brian', ' Holy Grail']) 

' <ulXli>Life of Brian</liXli>Holy Grail</liX/ul>' 


fo 1 *’ M r l eas y on your eye, but 
,he as tav- as your Web browser is 


The header () function lets you quickly format HTML headings at a selected level (with 2 as the default): 

»> header ("Welcome to my horne on the web") 

'<h2>Welcome to my horne on the web</h2>' 

»> header ("This is a sub-sub-sub-sub heading" , 5) 

'<h5>This is a sub-sub-sub-sub heading</h5>' 


Kothi^ too e*diti^ here, but it "eris 
pedted- Sa»»e <joes V here- 


as 


Last, but not least, the para () function encloses a chunk of text within HTML paragraph tags: 

»> para("Was it worth the wait? We hope it was. . .") 

'<p>Was it worth the wait? We hope it was...</p>' 
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controller code 


Control your code 

Your model code is ready, and you have a good idea of how the yate 
module can help you with your view code. It’s time to glue it all together 
with some controller code. 

First things first: you need to arrange your wedapp 5 s directory structure to 
help keep things organized. To be honest, anything goes here, although by 
giving it a little thought, you can enhance your ability to extend your webapp 
over time. Here 5 s one folder structure that Head First Labs recommends. 



You da» tali your -tof—levd 
folder anytWm^ you hke- " 



As well as ton-taiwmj the subf olders, this toldev- 
do»tai»s your webapps "mde^hW f'I* Y“ r 
"favido«.ito" s h le f d 

-I,- 4-UaJ- doesv>’t fit »eatW '"to o»e ot the 



Head on over to the 
Head First Python support 
website, download 
webapp.zip, and 
unpack it to your hard 
disk. 





A*y todt that you write -Pov- 

your webapp needs to reside i h a 

spedially »amed folder dalled "dqi- 
bih . J 


L-et’s keep the toadh s 
data files m a separate 
•folder by putti»3 ali 
the T)<T f iles m here- 





flK’ P^s, and so oj, p op 
■themmf» their ow» folder to help 
Kccp thihgs ov-gahiz^d. 



The templates that # 
^ da«»e with the yatepy 

doumload ta» 50 i» here- 
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web development 


C(M lets your web server rim programs 

The Gommon Gateway Interface (GGI) is an Internet Standard that allows for 
a web server to run a server-side program, known as a CGI script. 

Typically, GGI Scripts are placed inside a special folder called cgi-bin, so 
that the web server knows where to find them. On some operating Systems 
(most notably UNIX-styled Systems), GGI Scripts must be set to executable 
before the web server can execute them when responding to a web request. 


/K 


/VIove oy\ “tbis m 
a li-fctle bit 


So...to run my webapp, 

I need a web server with 
CGI enabled. 




AII webapps need to run on web servers. 

Practically every web server on the planet supports GGI. Whether 
your running Apache , IIS, nginx, Lighttpd, or any of the others, they 
ali support running GGI Scripts written in Python. 

But using one of these tools here is overkill. There 5 s no way the 
coach is going to agree to download, unpack, install, configure, 
and manage one of these industry heavyweights. 

As luck would have it, Python comes with its very own web server, 
included in the http . server library module. Gheck the 
contents of the webapp . zip download: it comes with a CGI- 
enabled web server called simplehttpd. py. 
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generate list 


Pisplay the list of athletes 

Let’s create a program called generate_list. py which, when executed 
by the web server, dynamically generates a HTML web page that looks 
something like this: 



3 paragvaph 



Thevcs oY\t v-adio 
button *Pov 

atblete- 


It wouldh t huirfc to 
add a bile io this 




AH of this is 
^oivtained withih 
WT/VJL fovr*. 


f\ “submit” button 


When your user selects an athlete by clicking on her radio button and clicking 
Select, a new web request is sent to the web server. This new web request 
contains data about which radio button was pressed, as well as the name of a CGI 
script to send the formas data to. 

Recall that all of your CGI Scripts need to reside in the egi-bin folder on 
your web server. With this in mind, let 5 s make sure your generate_lis t. py 
CGI script sends its data to another program called: 

cgi-bin/generate_timing_data. py 
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web development 



P00] puzzjc 


Your job is to take the code from the pool and 
place them into the blank lines in the CGI 
script. You may not use the same line 
of code more than once. Your goal is to 
make a CGI script that will generate a 
HTML page that matches the hand-drawn 
design from the previous page. 


fve stdV"tedi import athletemodel 

■fcWmy ^ import yate 

-fov you. import glob 


Th '^1 t "yate" 

e glob module leis you <^ev-y you> . 
operatmg sysie» a | ist J 4 »a»,es. 


data_files = glob.glob("data/*.txt") 

athletes = athletemodel.put to store(data files) 




L-ei’s add a lihk -to 
*thc boiiont oi "the 

generaied WTA]L 
P a 9c ibat takes 
youv user bo*«e. 


print(yate.include footer({"Home": "/index.html"})) 




Note: each thing fror 
the pool can be used 
once! 


r prlnt (yate.starf e . , 

P " K lBte ' - 'Coae " 11 t0 

' nich _athlete" Y llst of Afhi 

for each athlete in athletes: ’ at hletes fear-h 4., le tes")) 

^^ POnSe ° ) - Prlnt(yate — f-dl name) 


>r: 
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egi script 



Poo] pnzzjc ^o]ufioii 

Your job was to take the code from the pool 
and place them into the blank lines in the 
CGI script. You were not to use the same 
line of code more than once. Your goal 
was to make a CGI script that generates a 
HTML page that matehes the hand-drawn 
design. 


import athletemodel 
import yate 
import glob 



S-br-fc genevatma -the 

I J 

data_files = glob . glob ("data/* . txt") Providihg 3 k 

AUavs stari a appropHa-fce -ti-tlc. 

7 / - , ». athletes = athletemodel.put to store (data files) 

Ccmtcnt--fcffc W- *\ - - 

qi , Sprint (yate . start_response () ) 

vta ’ r J geheratma .—. k pav . a(V raph -teUin^ 

the +ov»>, ..i.-.. ... .. V ouv wset v»Wat to 

p\rovidihg -the har*e^ print (yate . start_form ( "generate_timing_data . py" ) ) ^ 

the sev-vev— print (yate .para ("Select an athlete from the list to work with:")) 

side pigrar* -to 

lihk -fco. ^. 

for each_athlete in athletes: 

print (yate . radio button ( "which athlete", athletes [each athlete] . name) ) 

^eneva-te a radio- ..-.“. c-yir-r . ”•••• 

wtbh Jfcr eath o( a £usbm 

^iLle-tes- ^ubmit buttoh. 

i m " ' ”' print(yate.include footer({"Home": "/index.html"})) 
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Cr* ft y pool- 






















web development 



What you need to do next depends on the operating 
system you’re running your web server on. 

. r • iJ Ifyou are running on Windows, stop reading right now and 
VvdlWl 11. proceed to the Test Drive. However, ifyou are running a Unix- 

based system (such as Linux, Mac OS X, or BSD) you need to 
do two things to prepare your CGI script for execution: 

1. Set the executable bit for your CGI using the chmod +x command. 

2. Add the foilowing line ofcode to the very top ofyour program: 

#! /usr/local/bin/python3 




'esT DriVq 


Fr<w Y OUV tevVmal Wmdow, bffe 

chmod +x generate list. 

py {p set the e*efcutable b.t- Yo«* 
need do this only onte- 


To test drive your CGI script, you need to have a web server up and running. The code to 
simplehttpd. py is included as part of the webapp . zip download. After you unpack the ZIP 
file, open a terminal window in the webapp folder and start your web server: 


File Edit Window Help WebServerOnWindows 


c: \Python31\python .exe simplehttpd.py 
Starting simple httpd on port: 8080 



Wse -this £or*r*ahd 
oh Mhdows-based 
sys-fcems. 


File Edit Window Help WebServerOnUNIX 



$ python3 simple_httpd.py 
Starting simple httpd on port: 8080 
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te st drive 



'qst DriVq, conriineD 


With your web server running, let's load up Coach Kelly's home page and get things going. YouVe 
started your web server running on port 8080 on your computer, so you need to use the following 
web address in your web browser: http : //localhost : 8080 . 




Welcome to Coach Kelly's Website 


(khttp://localhost:8080/ 


C (Q^ Google 


OD :::: Apple Yahoo! Coogle Maps YouTube Wikipedia Popular' 



Welcome to Coach Kelly’s Website. 

For now, ali that you’ll find here is my athlete’s timing data. Enjoy! 

See you on the track! 



The "tWmj 
da-ba" hY^cv-lmk is 
“fov Y ou 
-to dlidk Vt 




A 


The £oa£h’s home page 
appea« in your bv-owsev. 
Ii’s called "index.html" 

it is induded in the 
Vbapp.^ip" download. 


...and 70WV Nweb servev 
spvings into 1'te, 

(to the stveen) any and 
all >web ve<\«*ests that *t 

pv-o£esses. 



$ python3 simple_httpd.py 
Starting simple_httpd on port: 8080 

localhost - - [12/Sep/2010 14:30:03] "GET / HTTP/1.1" 200 - 

localhost - - [12/Sep/2010 14:30:03] "GET /coach.css HTTP/1.1" 200 - 

localhost - - [12/Sep/2010 14:30:03] "GET /images/coach-head.jpg HTTP/1.1" 200 - 

localhost - - [12/Sep/2010 14:30:03] "GET /favicon.ico HTTP/1.1" 200 - 


| Fii^J^7Tfiuuw"HeTp DisplayingHomePage 
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web development 


Sure enough, clicking on the horne page's link runs the generate_list. py program on the web 
server, which displays Coach Kelly's athletes as a list of radio buttons. 


* o O 


Coach Kelly's List of Athletes 


(k http://localhost :B0&Q/cgi-bin/generateJi5t.py 


C (Q, t Google 


PP ssss Apple Yahoo! Coogle Maps YouTube Wikipedia PopufarT 

Coach Kelly's List of Athletes 


Select an athlete from the list to work with: 

James Lee 
O Sarah Sweeney 
O Julie Jones 
O Mikey McManus 

Select 

Home 


| File Edit Window Help DisplayingHomePage 


$ python3 simple_httpd.py 
Starting simple_httpd on port: 8080 
localhost - - [12/Sep/2010 14:30:03] 
localhost - - [12/Sep/2010 14:30:03] 
localhost - - [12/Sep/2010 14:30:03] 
localhost - - [12/Sep/2010 14:30:03] 
localhost - - [12/Sep/2010 14:45:16] 


Lookmg goocL.-thff web 

page has been gexev-ated 
Coirv-ed-tly... 



...a«d yow web sevvev logs 
■the web re<\uest to w 

•the "genevatejist pv" 
C^l stvipt- 


"GET / HTTP/1.1" 200 - 

"GET /coach.css HTTP/1.1" 200 - 

"GET /images/coach-head.jpg HTTP/1.1" 200 - 

"GET /favicon.ico HTTP/1.1" 200 - 

"GET /cgi-bin/generate list.py HTTP/1.1" 200 



■ 




You can click the Home hyperlink to return to the coach 5 s home page, or 
select an athlete from the list (by clicking on their radio-button), before 
pressing the Select button to continue. 


Select an athlete and press Select. What happens? 
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no such egi script 


The dreaded 404 error! 

Whoops! Your web server has responded with a “404” error code, which is 
its way of telling you that something was wrong with your request. The web 
server is in fact telling you that it can’t locate the resource that your web 
browser requested, so it 5 s telling you that you made a mistake: 


^ O ^ Error response 


< 


+ 

\ _ 

http://localhost:S080/cgi-bin/cjenerate timing data C 

( Q? Google 


QQ :::: Apple Yahoo! Google Maps YouTube Wikipedia Popularv 


Error response 

Enor codc 404. 




Gheck the web server’s console window to confirm that your attempt to post your 
fornTs data to generate_timing_data . py resulted in failure. 

Which isn’t really that surprising seeing as you haveyet to write that code! So.. .things 
aren’t as bad as they first appear. The “404” error is exactly what you would expect 
to be displayed in this situation, so your generate_list. py GGI is working 
fine. What’s needed is the code to the other GGI script. 

If you create the required CGI script, you’ll be back on track. 
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web development 


Fireside Chats 



Tonighfs talk: To be CGI or not to be CGI, that is the question. 


A Python Program: 

Listen: yoffre really not all that different than me; 
you just work on a web server, whereas I can work 
anywhere. 

Special?!? But you only work on the Web, nowhere 
else. How’s that “special”? 


JVonsense! The truth is that you work only on the Web 
and break pretty quickly when used elsewhere. You 
dofft even have control over your own I/O. 


Like [sniggers] generating text in the form of 
HTML? ThaFs really taxing... 


Oh, get over yourself! Yoffre a regular program, 
just like me. I can generate HTML, too, I just 
choose not to. 

I guess so... 


Ummmm.. .1 guess so. 


A Python CGI Script: 


Yes. I like to think of myself as special. 


Because all the cool stuff works on the Web these 
days and Fm designed, optimized, tailored, and 
engineered for the Web. Because the Weffs a cool 
place, it follows that I must be cool, too. See: special. 


I dofft need control over my input and output. I 
have a friendly web server to take care of that for 
me. My input comes from the web server and my 
output goes to the web server. This arrangement 
allows me to concentrate on the important stuff. 

Smirk all you want; HTML makes the World Wide 
Web go around and Fm a master at generating it 
dynamically , on demand , and as needed. Without me, 
the Web would be a pretty static place. 


And if you did generate HTML, yoffd want it 
displayed somewhere.. .like in a browser? 

And to do that yoffd need to rely on the Services of 
a friendly web server, right? 

Which would make you a CGI script. So, yoffd be 
special , too. Q.E.D. 
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yet another egi script 


Create another Cfrl script 


Let 5 s take a moment to recall what is required from the generate_ 
timing_data . py GGI script. Based on your hand-drawn sketeh from 
earlier, your need to generate a new HTML page that contains the top three 
times for the selected athlete: 


TWis loob like a 
HTML headmg tov. 
tbe page, vigWt? 


lis pn-obably a jood 
'dea io add a -ti-tle 

io -tbis page- 





da*ta -Cor Sav-ah* 


This miglvt be bes-t 
rehdeved as a* 

uho\rdcv"cd HT/WL Ii si 



Lc-fs ihdude 
"the ah-fcle-fce^s 
-Pull hame and 
hev-e. 


Home 


Selefrt amotbev athleie- 



Tno bwerlmb:<«e h lV,e ‘Tk 
page, Ue«as the oibev reW -to tbe 


Put how do you kwow which athlete is selected? 

When you click on a radio-button and then press the Select button, a new 
web request is sent to the server. The web request identifies the GGI script 
to exeeute (in this case, that 5 s generate_timing_data . py), together 
with the fornis data. The web server arranges to send the form 5 s data to your 
GGI script as its input. Within your code, you can access the form data using 
Python’s egi module, which is part of the Standard library: 


|mf0V“t ‘the 

w t<y * liWav-y- 



$rab ali ot the data 
a»d put it i» a didtionavy. 


import egi 

form_data = egi.FieldStorage() 

athlete_name = form_data[’which athlete’].value 


/\ddess a hamed piede o£ da*ta -Pyom *the -Povm s data- 
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web development 



VVv-i-tc "bV»c 6odc 



lt's time to exercise your newly acquired web-coding chops. Grab 
your pencil and write the code for the generate_timing data . 
py CGI script. It's not too different from the generate_list. py 
code, so you should be able to reuse a lot of your existing code. 
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revised script 



lt's time to exercise your newly acquired web-coding chops. You 
were to grab your pencil and write the code for the generate_ 
timing_data . py CGI script. It's not too different from the 
generate_list. py code, so you should be able to reuse a lot of 
your existing code. 


it 2 ! /usr/lodal/bm/ 1 pytbor3 




-This Iine is needed on Uni*-based systems cmly. 


impo\rt Cy 


|mpovt tbe \ 

libvavics imPo\rt atbletemodel 

modules you / 1 

mte*d -to use- y impo\rt yate 


the data . 

-Pirom the — atbletes =■ atblete model- 3 et_^vom_stov-e 0 
model- . 


W/bi eb C" p . 

atblete^ \ to\rm__data =■ dgi.pieldStov-a^eO 

vou f atblete__Kame — £ov-m__datarwbieb__atblete ; 3.value 

litb? . 


Wothing 
*ew here 
or here. 



prmt(yate-start_respoKseO) 

py'mt^yate.'mdlude_beadev-r ; Coadb ^elly*s TWw$ Data ; 0) 


prmtfyate-beaderrT^blete: M + atblete *ame + > \ V0b : ” + L ^ _, f , t ^ € 

a*d p£>B- 


atbletesCatblete r\ame3-dob + V)) 


prirrKyate.para("The top times for tbis atblete are ")) ^ ^ ^ee list mto 

pv-'mt^yate u__listfatbletesCatblete_hame3.top30)) ^ a* urordered HTML*bst 


Tbe bottom of 
tbis u/eb pa$e 
has two Imks- 


pv'mt^yate 'mdlude_^ootev-({ w ttome w : V'mde%-btm| w , 

w £eledt a»otber atblete” : ;; ^e ^ e \rate 1 i st - py) 


\ A li»k back io ihe pveviow 
C$l sevipt 


246 Chapter 7 


































web development 



'esT DriHq 


No*tc: |*f you ave oy\ a Uni%-based system, dor! t 'Cov^et 
*to add u chmod +x generate_timing_data. py “to 
se*t *the e%e£u*lable bit 


Your web server should stili be running from earlier. If it isn't, start it again. In your web browser, 
return to the coach's home page, then select the hyperlink to display the list of athletes, select Sarah, 
and then press the button. 


Tbis all 
iooks 0^- 



phooey/ 

Soinac-thihg^s ho-fc 
«\ui-tc vighi heve. 
Wheve s Savah s 
*fc°P thvce times? 




Coach Kelly's Timing Data 


(k httpy/localho5t:BQSQ/cgi-bin/generate_tirning_data C i.Q, T Google 




QQ :::: Apple Yahoo! Coogle Maps YouTube Wikipedia Popular 1 


CoachKell y 's Timin g Data 

Athlete: Sarah Sweeney, DOS: 2002-6-17 


The top times for thfs athlete are: 




e 


poes the 
web sevN/ev s 
\om^ 
’m?ovmatioir\ 

-teli you 
arytbmj? 



I File Edit Window Help HoustonWeHaveAProblem 


$ python3 simple_httpd.py 
Starting simple_httpd on port: 8080 

localhost - - [12/Sep/2010 14:30:03] "GET / HTTP/1.1" 200 - 

localhost - - [12/Sep/2010 14:30:03] "GET /coach.ess HTTP/1.1" 200 - 

localhost - - [12/Sep/2010 14:30:03] "GET /images/coach-head.jpg HTTP/1.1" 200 - 

localhost - - [12/Sep/2010 14:30:03] "GET /favicon.ico HTTP/1.1" 200 - 

localhost - - [12/Sep/2010 14:45:16] "GET /cgi-bin/generate_list.py HTTP/1.1" 200 - 

localhost - - [12/Sep/2010 16:12:27] "GET /cgi-bin/generate_list.py HTTP/1.1" 200 - 

localhost - - [12/Sep/2010 16:12:29] "POST /cgi-bin/generate_timing_data.py HTTP/1.1" 200 - 

Traceback (most recent call last): 

File "/Users/barryp/HeadFirstPython/chapter7/cgi-bin/generate_timing_data.py", line 21, in <module> 
print(yate.u_list(athletes[athlete_name].top3())) 

TypeError: 'list' object is not callable 

localhost - - [12/Sep/2010 16:12:29] CGI script exit status 0x100 


Your CGI has suffered from a TypeError exception, but other than looking at the web 
server 5 s logging screen, it’s not ciear on the web browser screen that anything has gone wrong. 



What do you think is the problem here? Take a moment 
to study the error message before flipping the page. 
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track down egi errors 


Enable C(M tracking to Help with errors 


The GGI Standard dictates that any output generated by a server-side 


program (your GGI script) should be captured by the web server and sent 
to the waiting web browser. Specifically, anything sent to STDOUT (Standard 
output) is captured. 

When your GGI script raises an exception, Python arranges for the error 
message to display on STDERR (Standard error). The GGI mechanism is 
programmed to ignore this output because all it wants is the GGI scripfs 
Standard output. 



This behavior is fine when the webapp is deployed, but not when it’s being 
developed. Wouldn’t it be useful to see the details of the exception in the 
browser window, as opposed to constantly having to jump to the web server’s 
logging screen? 


Well.. .guess what? Python’s Standard library comes with a GGI tracking 
module (called egitb) that, when enabled, arranges for detailed error 
messages to appear in your web browser. These messages can help you work 
out where your GGI has gone wrong. When you Ve fixed the error and your 
GGI is working well, simply switch off GGI tracking: 


import egitb 
egitb.enable() 



xc^hhology 


hdd -fchese -fcwo |*, hCS 

sUU of yo U r C$l sCvip-ts io 
enable Pyihons C^( -tvadking 
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web development 



Wo\^ Look &W 
o$ “tWts dc-ta'»l 


Add the two CGI tracking lines of code near the top of your generate_timing_data . py CGI 
script. Press the Back button on your web browser and press the Select button again. Let's see what 
happens this time. 


O 


Coach Kelly's Timing Data 


(K http://localhost:e0S0/cgi-bin/generate_timingi_data. py 


m 


Apple Yahoo! Coogle Maps YouTube Wikipedia Popular’ 



Coach Kelly's Timing Data 


Athlete: Sarah Sweeney, DOB: 2002-6-17 


The top times for thfs athlete are: 


TypeError 


Pythc 

/Li t> rary/Frame wo rks/Pytho n .frame wo rkA/e rsio ns/3.1 /Reso u rces/Pytho n .ap p/Co nte ntsTMadDS 

Sun Sep 1220:13 


A probi em occurred In a Python script. Here Is the sequence of functlon calls leadlng up to the error. In the order they 
occurred. 


/Users/barryp/HeadFirstPython/chapter7/cgi-bin/generate_tiniing_data.py In {} 


19 athletes[athlete_name]*dob + ".")) 

20 print{yate .para{ "The top times for this athlete are:"}) 


> 21 print{yate.u_list{athletes[athlete_name]*top3())) 

2 2 print{yate. include_£ooter { { "Horne" i 11 /index, html" , 

21 ''Select another athlete": "generatV_list .py" 1*) ) 


buiitin print = <buSlt-Sn functlon print>, yate = <modu!e 'yate'from '/Users/barn\p/HeadFlrstPython/chapter7/cgl- 
bin/yate.py'>, yate.ujist = <functlon u_list>, athletas = {'James Lee': ['2-34', ':|:21', '2.34', '2.45', '3.01', '2:01', 
'2:01', '3:10', '2-22', '2-01', '2.01', '2:16'], 'Julie Jones': ['2.59', '2.11', '2:11', '2:23', '3-10', '2-23', '3:10', '3.21', '3-21', 
'3.01', '3.02', '2:59'], 'Mikey McManus': ['2:22', '3.01', '3:01', '3.02', '3:02', '3.02', '3(22', '2.49', '2:38', '2:40', '2.22', '2 
31'], 'Sarah Sweeney': ['2:58', '2.58L^3a^£=25^2-55', '2:54', '2.18', '2:55', '2:5/5', '2:22', '2-21', '2.22']}, 
athlete_name = 'Sarah Sweeney', SLtop3 unde fine 


TypeError: 'list' object Is not callabfe 

args = ( n llst r object Is not callable",) 

wlth_traceback = <bulft-in method wlth_traceback of TypeError object> 


Notice that the CGI tracking module tries to identify 
exactly where the problem with your code lies. 



Whafcs this? {op30 is 

«nde-fined?/? 
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small fix, big difference 


A small change can make all the difference 

The GGI tracking output indicates an error with the use of the top3 () method 
from the AthleteList code. 

A quick review of the code to the AthleteList class uncovers the source of the 
error: the top3 () method has been redesignated as a class property. 



This deeovaiov allows y ou io attcss ihe daia 

veiuvned by "iop30" as if i 

Y ^ w 1 £b “ 


@property 
def top3(self): 

return(sorted(set([self.sanitize(t) for t in self]))[0:3]) 


The use of the @property decorator allows the top3 () method to appear 
like an attribute to users of the class. So, instead of calling the top3 () 
method like this: 



h ^e-fchod tali 
always Keeds -the 
pav-Ch-thcscs... 


Treat the top3 () method as if it was another class attribute, and call it like 
this: 



If s a small change, but if s an important one 

When a change is made to the way a class is used, you need to be careful to 
consider what impact the change has on existing programs, both yours and 
those written by others. 

At the moment, you are the only one using the AthleteList class, so it’s 
not a big deal to fix this. But imagine if thousands of programmers were 
using and relying on your code... 


..unless ibe meibod is^detlaved 
io be a» '%v°?evty’> m ' wW,tV ' 
tase yavenibeses ave NOT 
v-e^uired- 


Let’s fix your CGI script and try agam. 
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Make the small edit to your code to remove the brackets from the call to the top3 () method, press 
your web browser's Back button, and press the Select button one last time. 


VVhoo ViooJ This 
■bime bhe scictbcd 

abhlebe s data »s 
dis^laycd o* sdv-ee*- 

U\u> eh? 


r) o 


Coach Kelly's Timing Data 


(K http://locaihost:e0S0/cgi-bin/generate_iimIngi_data C ' Q, T Google 




CP 


Apple Yahoo! Coogle Maps YouTube Wikipedia Popular 1 


CoachKell y '5 Timin g Data 


Athlete: Sarah Sweeney, DOB: 2002-6-17 


The top times for this athiete are: 



Home Select another athlete 


Now that you v e ^ 
solved -feha-t 
PToblcm, bc SUTC 

to swibch otf 
C$l trackihg. 




tiiereiare no 

Dumb QiiestiQns 


What happens if the coach recruits new athletes? 

AII Coach Kelly needs do is create a new text file similar to the 
others, and your webapp handles the rest by dynamically including 
the new athlete the next time your webapp runs, which occurs when 
someone clicks on the home page’s “timing data” hyperlink. 


Q.: Shouldn’t the server’s data be in a database as opposed 
to a pickle? Surely that would be better, right? 


In this case, it’s probably overkill to use a database, but it might 
be worth considering sometime in the future. 
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successful webapp 


Your webapp'$ a hit! 





Hey, this is super cool 
and the coach is right... 
if we train hard, we can 
improve on these times! 


By moving your program to the Web, youVe made it a no-brainer for Coach 
Kelly to share his data with not only his athletes, but with anyone else that 
needs to access his data. 

By conforming to the MVC pattern and using CGI, youVe built a webapp in 
such a way that it’s easy to extend as new requirements are identified. 


Congratulations! You’re a web developer. 
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Your Pythow Toolbox 

You’ve got Chapter 7 under your 
belt and you’ve added some key 
Python techiques to your toolbox. 


Pyitioh Lj 


i - a <wat» ik . , 

leh Y* arr-ah-e , ,, , 3i 

a PP«*- as i-f j^ is , . a “ h> 

t,sa attribui. 


'hjo 


VVA Umy 


We .. st „t W tt* 

I lUp yicV) SCWCV-. 

Woviscv ±0 the ^ 

» scv ^ £voir* *tV*c ^ 

# U YicV) V"CS^or\SC vc osor\SC 

se ,v e , to *e ** WoV,SeV ? 

•to a 'MeV) vc<\<Aest- 

"C6I” - the Co"'">o" ^ 

• r w/V» allovs a *eb server to 
1^3«, ^ a» 0 

, SC yver-s'die froya»- 

vw a serve» 

.. . l-’ , a * 0 thev «aw>e tor a 

# *C$I *t«?t - 

sewer-s\de yroyra^ 




BULLET POINTS 


The Model-View-Controller pattern lets 
you design and build a webapp in a 
maintainable way. 

The model Stores your webapp’s data. 

The view displays your webapp’s user 
interface. 

The controller glues everything together 
with programmed logic. 

The Standard library string module 
includes a class called Template, 
which supports simple string substitutions. 

The Standard library http. server 
module can be used to build a simple web 
server in Python. 

The Standard library egi module 
provides support for writing CGI Scripts. 

The Standard library glob module is 
great for working with lists of filenames. 

Set the executable bit with the chmod 
+x command on Linux and Mac OS X. 

The Standard library egitb module, 
when enabled, lets you see CGI coding 
errors within your browser. 

Use egitb . enable () to switch on 
CGI tracking in your CGI code. 


Use egi . FieldStorage () to 

access data sent to a web server as part 
of a web request; the data arrives as a 
Python dictionary. 
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8 mobfle cipp deVelopment 


^ Small devices + 



Th 


had 


be+ter 


be 


is 


a 


smartphone running 
Honeycomb or Mr. Smooth 
is history! 




Putting your data on the Web opens up all types of possibilities. 

Not only can anyone from anywhere interact with your webapp, but they are increasingly 
doing so from a collectiori of diverse computing devices: PCs, laptops, tablets, palmtops, 
and even mobile phones. And it’s not just humans interacting with your webapp that 
you have to support and worry about: bots are small programs that can automate web 
interactions and typically want your data, not your human-friendly HTML. In this chapter, 
you exploit Python on Coach Kelly’s mobile phone to write an app that interacts with your 
webapp’s data. 
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going mobile 


The world is gettiwg swaller 

Coach Kelly is continuing to use his webapp every day, but he 5 s having a 
problem with his new smartphone. 



There'$ more thaw just desktop computers out there. 

Who knew that your users would try to interact with your webapp using 
something other than a desktop computer or laptop? 

It’s a diverse computing environment out there. 
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Coach Kelly is <m Android 

The coach has a lovely new smartphone that’s running Google’s Android 
operating system. Sure enough, when you check it out, the webapp is way too 
small and not much use on the coach’s three-inch screen: 




Obviously, the coach needs to access his data and run his webapp 
on his phone.. .but what’s the best way to do this if not through the 
phone 5 s browser? 



0 





Open your web browser on your desktop computer (or phone) 
and enter "Python for Android" into your favorite search engine. 
Make a note in the space below of the most promising site from 
your search results: 
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scripting layer for android 


^ l 




-tWis i\\t 
U “foiAttd? 



Solution 


You were to open your web browser on your desktop computer 
(or phone) and enter "Python for Android" into your favorite 
search engine. You were then to make a note in the space below 
of the most promising site from your search results: 


lvt*tp : //£ode.^oo^lc dom/p/ahdroid-sdv-iptm^ home o-f *the pv-oje£*t) 


Kun Python on the coach'$ smartphone 

A quick search of the Web uncovers a pleasent surprise: Python runs on Android. 

At least a ver sion of Python runs on Android. A project called Scripting Layer 
for Android (SL4A) provides technology to let you run Python on any Android 
device. But there’s a catch. 



O 






A python 


Ttf 


just checked 
the SL4A website, and 
it looks like it supports 
Python 2.6.2, not Python 3. 
Phooey! 





m s n f 


B B » d 

0>ii/ 

er 



Yes. SL4A ships with Python 2, not 3. 

Python 3, this book 5 s preference, is the best version of 
Python yet, but it achieves its greatness at the cost of a 
lack of backward compatibility. There’s some stuff in 
3 that will never work in 2 and vice versa. 

Is this fact alone a show-stopper ? 
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PowT worry about Python l 

The fact that Python 2 is available for Android and youVe learned Python 
3 in this book is nothing to lose sleep over. Python 2 is stili Python , and the 
differences between Python 2 and Python 3 are easy to manage. 

Think about your webapp for a minute. Right now, the model, view, and 
controller code resides on the web server, which is running Python 3. 



0 


Youv- web bvowsev 




The Internet 


l m i um 


Web 

Server 


If you move the user interaction to the smartphone, the model and some 
of the controller code stay on the server (and continue to run on Python 
3), whereas the view code and the rest of the controller code move to the 
smartphone, where they need to be rewritten to run on Python 2. 


AII o-f yow webapp 

todt \ruhs hev-e. 











The Internet 


Hal-f youv- webapp's 
£ode iruhs hev*e... 





Web 

Server 


...and *tbe o-tbev 
hal-f 'runs beve. 


I 


/ / 1 
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android sdk 


Set up your development environment 

Understandably, the coach won’t let your have his phone to work on until 
you have something that works. Thankfully, Google provides a cross-platform 
Android emulator that lets you develop for the phone as needed, even 
though you don’t own any hardware. 


Download the Software Pevelopwewt Kit (SPK) 

Let 5 s get started developing for Android. Visit this website and download the 
SDK for your computer and operating system: 


http://developer. android. com/sdk/ index.html 



Follow along with 
these instructions to 
ensure you have your 
Android development 
environment correctly set 
up on your computer. 


« o 


Android SDK | Android Developers 



(e) 


) i 


r 


http://developer.android.com/sdk/index.html 


^ (tfK Google 


CD 

Q/ 


BBC News 3* RT£ News i?* G-Mail 0'Reilly Radar Head First Labs LJ - Front LJ - News S* Safari GNews Ireland S* Twitter 
Android SDK | Android Developers + 


CIC0300 

d 


<3 English ij 


id.com 


nopers 


Home 


Dev Guide 

Reference 


Resources 


Videos 


search developer docs 


Blog 


Search 


Android SDK Starter Package 

Installing the SDK 


Downloadable SDK Components 

Adding SDK Components 

Android 2.2 Platform 
Android 2.1 Platform 
Android 1.6 Platform 
Android 1.5 Platform 
Older Platforms 

SDK Tools, r7 "■"* 

USB Driver for Windows, r3 

ADT Plugln for Ecllpse 

ADT 0.9.8 newl 

Native Development Tools 

Android NDK, r4b new! 

More Information 

SDK System Requirements 


Download the Android SDK 


Welcome Developers! If you are new to the Android SDK, please read the Quick Start . below, for an overview of how to 
install and set up the SDK. 

If you are already using the Android SDK and would like to update to the latest tools or platforms, please use the 
Android SDK and AVD Manager to get the components, rather than downloading a new SDK package. 


Platform 

Package 

Size 

MD5 Checksum 

Windows 

android-sdk r07-windows.ziD 

23669664 bytes 

69c40c2d2e408b623156934f9ae574f0 

Mac OS X (inteh android-sdk r07-mac x86.ziD 

19229546 bytes 0f330ed3ebb36786faf6dc72b8acf819 

Linux (i386) 

android-sdk r07-linux x86.taz 

17114517 bytes 

ei 0c75da3d1 aal 47ddd4a5c58bfc3646 


Done 




The Android 
websrte 




Despite what this website might look like it 5 s telling you, you do not need to 
install Eclipse to run the Android emulator. However, you do need to have a 
Java Runtime Environment installed. If you are unsure about this, don’t worry: 
the Android emulator will advise your best course of action if it spots that 
Java is missing. 


No-t* This is hovi the Andvoid SpK 
dovmload W looks at the W 
this y/vitivA It look a httle 

dVfWb tov you. No 'Movvies: juSX. 

doumload the latest ve«.o« ok the 
SPK. 
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Configure the SPK and emulator 

You need to do two things to configure the SDK and emulator: add an Android 
Platform and create an Android Virtual Devie e (known as an TDD). 

Add an Android platform 

The coach is running Android 2.2 on his phone, so let’s add a 2.2 platform 
to mimic this setup. Open up the Android SDK and AVD Manager tool, 
select Available Packages, and pick 2.2 for installation. 



The Attdvoid dovmload dorrtams 
a -folder talled R*» w 

“android” pv-oyar» Wrthm “bhis 

-foldev. 




Android SDK and AVD Manager 


Virtual Devices 
Installed Packages 
Available Packages 
Settings 
About 


Sites, Packages and Archives 

▼ ® https://dl-ssl.google.com/android/repository/repository.xml 
► Documentation for Android SDK, API 8, revision 1 


— i 

□ 

n 


SDK Platform Android 2.2, API 8, revision 2 


► 'f' SDK Platform Android 2.1, API 7, revision 2 

► 'i' SDK Platform Android 1.6, API 4, revision 3 

► 'i' SDK Platform Android 1.5, API 3, revision 4 

► (2) Samples for SDK API 8, revision 1 

► (2) Samples for SDK API 7, revision 1 

► '|f.Coogle APIs by Coogle Inc., Android API 8, revision 2 

► '^.Coogle APIs by Coogle Inc., Android API 7, revision 1 

► '^.Coogle APIs by Coogle Inc., Android API 4, revision 2 

► '|f.Coogle APIs by Coogle Inc., Android API 3, revision 3 

► fiiMarket Licensing package, revision 1 



Add Add-on Site... j Delete Add-on Site... ^ Displ ( RefreshJ) (jnstall Selected 


C.. 

____ J ► 

Description 

Android SDK Platform 2.2_rl 


Revision 2 



Create a new Android Virtual Pevice (AVP) 

With the 2.2 platform downloaded and installed, create a new Android 
Virtual Device. 

f* Create new Android Virtual Device (AVD) 

£jive '/ouv ^ 

and seletl a tavjet 


Name: droid2.2 


Target: | Android 2.2 - API Level 8 

SD Card: 

© Size: 512 



MiB 


Skin: 





0 File: V\ 







© Built-in: Default (HVCA) T) 



Scb siz^ o-p 
viv-tual SD^ard- 5/ 2 is 
r *°' r £ t^an Choujh. 


Hardware: 




Property Value 

New... 

Abstracted LCD density 160 

Delete 





Clitk o*> 
"Create AVP”- 


This is -fche ohly vevsion ot 
the £t>e that you need- 


TV.s *i 9 ht take a r»i hu te ov- two, 
depending oh the speed ot yow- 
hctwoyrk donne^tion. 



Your AVD is 

a simulated 
Android plione. 
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emulate sl4a 


Install and configure Android Scripting 

With the emulator ready, use the AVD Manager to start your 2.2 device. Click 
on the emulator 5 s browser (the little globe), surf to this web address: 

http://code.google. com/p/android-scripting 


These mstrud-tiohs work on a 'Veal" 

( P W, ioo. Xsi be sure io enable 

sow *es" io allow ior non- 
/Watrket applicatior doumloads. 


and tap on the “boxed” bar code near the bottom of the page: 


”°7 won 7 j + >i your emulator a 

-7 ,hute ~ ^ s ^rt The emulator is 
slowe^ than the aetual phone 


0* -the emula^V) 
-tap ov\ 

u bo%ed tav dodc 
-to start -the 
£L-A“A dovmload- 



When the download completes, select the emulatobs Menu button —► More 
Downloads, and then tap on the sl4a_r2 . apk file to install the SL4A 
package on the emulator. When the install completes, tap Done. 


The version available io you mi$ht be 
diWerent, Wfc dont the 

la-tcsi vclcasc- 



o Today 


sl4a_r2.apk 

android-scripting.googlecode.com 
821KB Download com-plete. ? 


The version you see «ntyht b« deterent than 
this. Pont 'norrf, yours in the "*>st veten . 
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Add Pythow to your SL4A iwstallatiow 



Filename 


idIBG 4 15 


python for android rl.apl 

sl4a r2.apk 

821KB OownlMd amplet* 


14/10 


Return to the emulatoks web browser, double-tap on the screen to zoom in, 
and select the Downloads tab. Double-tap again and tap the following link: 

python_f or_android_rl. apk ___— 

Tap the download link, and tap on the package name to download it. Select 
Menu —►More —► Downloads, and tap on the newly downloaded package. 


/\aa'm, -the vev-sio» you see 
be dVtWfc tha» this. Select the 

mosb 'rttCK t tile- 

A \ \ 


4:14 rv 




SSS*.drod?.? 


Summa 

sl 4 a_ir 2 .c 

rhino_fo 

Featured 

pytho n_ 
Featured 

perl_for. 
Featured 


A 

python for android rl.apk 


sru 


1 Checksum: b213ce6fGO1625e07468fc0a4t 
Use the SHA1 checksum shown to verify fil 

rity- 


a nd ro id- scripti ng 

OT.cf.fijf ijrtf tor jt/wft-.atfGf ff f.pf.njj 

erojt-tc «an»* Deimloidi Mrihl Isi- 
idlCh [jrif-ldomltudi ~w f* * 


android rl.apk 

di----~ — 


SSS4 dro^2 2 


=n»M<a 




rhino_for_androfcl_r1 ayK 
p yt bo n_f o r_a n d ro idjr 1. a o k 


peri for android rl.apk 


uajGr_*rKHo 

lwhred 


l*uby_r«5r_jnilr 




dl :• 42 


, python for androni.‘1 i>p'< 


uierume ■ 

ifinflliw JinA-3*l. 1 


liflrtin Im Jrdrjrf il.Apfr 


i imU r_f<in’U! jtfifc 


• i- 

MjjJ 

■Pirtfi I 

Funj-K 


' I * J kHJftr-j«to>_<LrJ .»j>» 


i<h»ik-u.M- h: '.^'nji t;vc'4i-iitc*4 
UW Ch* 1MA1 <**c«ujm ttawfn IO verify fi 
[nty 


MI 'IU 


S . 22 D l 21 

7 *Ta 

(A «D 

40* 


□ i 4 n< 



pyth on_for_a n d roi d_r1. a pk 

f android-scrlptlng.googlecode.com 
30.97KB Download complete. 9t 

rp 

I sl4a_r2.apk 

f android-scriptlng.googlecode.com 
821KB Download complete. 9; 



The Python for Android app runs. When you are ready, tap Open -> Install 
to complete the installation. This downloads, extracts, and installs the Python 
support files for Android, which can take a few minutes to complete. When 
it does, Python 2.6.2 and Python for Android are installed on your emulator 
and ready for action. 


This last bi-fc is veally 

impoirbh-t. 


Let’s confirm everything is working with a quick test. 


you are here ► 


263 



























































































your script on android 


Tcst Python on Android 

Return to your emulator 5 s main screen and find an app called SL4A added to your list 
of app icons. Tap this app to display the list of Python Scripts preinstalled with Python 
for Android. Simplty tap on any script name to execute it: 



SSS4 droid2.2 


-r\ w _ )} 

TnC mCnu 
louUon- 


Here’s a four-line Python script that you can create to test your installation. 
Let’s call this script mydroidtest. py: 


Be sure to set 
the SL4A 
rotation mode 
to automatic. 


Your screen might 
switch to landscape by default 
the first time you run a script. 
To fix this, choose Menu * 
Preferences, scroll down to 
Rotation mode, and set its 
value to Automatic. 


Take your Android ewulator for a spin 


ttBIl]® 8:38 pm II 


Scripts 
A w 


5L4A r2 


fgi bluetooth_chat.py 
hello_world.py 
•6* notify_weather.py 
say.chat.py 
ijl sayjime.py 
say_weather.py 
$ speak.py 
$ take_picture.py 
$ test.py 
fjl weather.py 


1 

2 i 

3 

4 

5 3 

6 

7 i 

8 

9 

° 

Q 

w 

E 

R 

T U 

y y 

u 

I 

0 j 

P 

A 

s 

D 

F 

G 

H 

J 

K 

L 

pu 

<xj 

❖ 

z 

X 

C 

V 

B 

N 

M 

. 

H 


SYM 

@ 

L _ l 

/ 

t 



Import “the “android 
library and dreate a ne*/ 
ayy objeet instande- 



Create an appropviate ^2* 
message and display it on 
screen. 



import android 

app = android.Android() 

msg = "Helio from Head First Python on Android" 
app.makeToast(msg) 


To transfer your script to the emulator, you need to copy it to the emulator 5 s 
virtual SD card. Another program within the tools folder called adb helps 
with this: 


the emulator. 






Your script should now appear on the list of Scripts available to SL4A. 
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Let's confirm that your Android setup is working. With the SL4A app open, simply tap on your script's 
name to run it, and then click the run wheel from the menu. 



Clitk your app 


■••the» ditk -the 
Vuh wheel." 




| mydroidtest.py started. 


Scripts 




5554:droid2.2 



Scripts 


•gifindj 


I§lget2ir 


hello_ 


•2® mara 


ffijd 5:03 pm 


mydroidtest.py 




m 

-■ 

m 1 


5554:droid2.2 


«S» bluetooth_chat.py 
v coachapp-KEEPER.py 
«8» coachapp.py 
«g» findjt.py 
« 8 » get2inputsapp.py 
« 8 » hello_world.py 
«8» marathonapp.py 
m vdrQidtest .pv 

Helio from Head First Python on Android 


MENU 


11 2 

3| 

4 

5 

6 

| 

8 

9 

0 

n w 

E 

R 

T 

Y 

U 

I 

0 

pM 

■ 

D 

F 

G 

H 

j 

K 


^el 

X 

C 

V 

B 

N 

M 

. 


@ 

, _ , 

/ i 

, 



1 

2 

3jj| 

4 

5 

6 

a 

8 

9 

0 

Q 

W 

E 

R 

T 

Y 

U*j 

I 

0 

P 

A 

s 

D 

F 

G | 

H 

j 

K 

HtMl 
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z 

X 

C 

V 

B 

N 

M 



B 

SYM 

@ 

■ l 

7 i 

, 



/W theres your messay. It 
wovksj 


Your Android 
emulator with SL4A 
is working, and it’s 
running your Python 
code. 
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what to do? 


Pefine your app $ rcqoircmewts 

Let’s think a little bit about what your Android app needs to do. 



Frank: Well.. .first off, the view code no longer has to generate HTML, 
so that makes things interesting. 

JM: In fact, you need the web server only to supply your data on 
request, not all that generated HTML. 

Joe: Ah ha! Fve solved it. Just send the pickle with all the data from the 
server to the Android phone. It can’t be all that hard, can it? 

Jill: Sorry, guys, that’11 cause problems. The pickle format used by 
Python 3 is incompatible with Python 2. You’11 certainly be able to send 
the pickle to the phone, but the phone’s Python won’t be able to work 
with the data in the pickle. 

Frank: Darn...what are our options, then? Plain data? 

Joe: Hey, good idea: just send the data as one big string and parse it on 
the phone. Sounds like a workable solution, right? 

JM: No, that 5 s a potential disaster, because you never know in what 
format that stringed data will arrive. You need an data interchange format , 
something like XML or JSON. 

Frank: Hmm.. .Fve heard XML is a hound to work with.. .and it’s 
probably overkill for this simple app. WhaPs the deal with JSON? 

Joe: Yes, of course, I keep hearing about JSON. I think they use it in 
lots of different places on the Web, especially with AJAX. 

Frank: Oh, dear.. .pickle, XML, JSON, and now AJAX.. .1 think my 
brain might just explode here. 

JM: Never worry, you only need to know JSON. In fact, you don’t even 
need to worry about understandingJSON at all; you just need to know 
how to use it. And, guess what? JSON comes Standard with Python 
2 and with Python 3.. .and the format is compatible. So, we can use 
JSON on the web server and on the phone. 

Frank & Joe: Bonus! ThaFs the type of technology we like! 
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This week’s interview: 

The Data Interchange Lowdown 


Head First: Helio, JSON. Thanks for agreeing to 
talk to us today. 

JSON: No problem. Always willing to play my part 
in whatever way I can. 

Head First: And what is that, exactly? 

JSON: Oh, I’m just one of the most widely used 
data interchange formats on the Web. When you 
need to transfer data over the Internet, you can rely 
on me. And, of course, you’11 find me everywhere. 

Head First: Why ’s that? 

JSON: Well.. .iPs really to do with my name. The 
“JS” in JSON stands for c JavaScript” and the “ON” 
stands for “Object Notation.” See? 

Head First: Uh...F m not quite with you. 

JSON: I’m JavaScripPs object notation, which 
means Fm everywhere. 

Head First: Sorry, but you Ve comple tely lost me. 

JSON: The first two letters are the key ones: I’m 
a JavaScript Standard, which means youdl find me 
everywhere JavaScript is.. .which means Fm in every 
major web browser on the planet. 

Head First: WhaFs that got to do with Python? 

JSON: ThaPs where the other two letters come 
into play. Because I was initially designed to allow 
JavaScript data objects to be transferred from one 
JavaScript program to another, Fve been extended 
to allow objects to be transferred regardless of what 
programming language is used to create the data. 

By using the JSON library provided by your favorite 
programming language, you can create data that 
is interchangeable. If you can read a JSON data 
stream, you can recreate data as you see fit. 

Head First: So I could take an object in, say, 


Python, use JSON to convert it to JSON 5 s object 
notation, and then send the converted data to 
another computer running a program written in C#? 

JSON: And as long as C# has a JSON library, you 
can recreate the Python data as G# data. Neat, eh? 

Head First: Yes, that sounds interesting...only 
[winks] why would anyone in their right mind want 
to program in G#? 

JSON: [laughs] Oh, come on now: be nice. There’s 
plenty of reasons to use different programming 
languages for different reasons. 

Head First: Which goes some of the way to explain 
why we have so many great programming tities, like 
Head First G#, Head First Java, Head First PHP and 
MySQL , Head First Rails , and Head First JavaScript. 

JSON: Was that a shameless, self-serving plug? 

Head First: You know something.. .1 think it might 
well have been! [laughs]. 

JSON: [laughs] Yes, it pays to advertise. 

Head First: And to share data, right? 

JSON: Yes! And thaFs exactly my point: when you 
need a language-neutral data interchange format that is 
easy to work with, iPs hard to pass me by. 

Head First: But how can you be “language neutral” 
when you have JavaScript in your name? 

JSON: Oh, that 5 s just my name. IPs what they 
called me when the only language I supported was 
JavaScript, and it kinda stuck. 

Head First: So they should really call you 
something else, then? 

JSON: Yes, but “WorksWithEveryProgramming 
LanguageUnderTheSunlncludingPythonObject 
Notation” doesn’t have quite the same ring to it! 
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leaving pickle on the piate 




You are not exactly “abandoning” pickle. 

The JSON technology is a better fit here for a number 
of reasons. First of all, it 5 s a text-based format, so 
it fits better with the way the Web works. Second, it 5 s 
a Standard that works the same on Python 2 and 
Python 3, so there are no compatibility issues. And 
third, because JSON is language-neutral, you open 
up the possibility of other web tools written in other 
programming languages interacting with your server. 

If you use pickle her e,you lose all this. 
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«* / An IDLE Session 

r 


JSON is an established web Standard that comes preinstalled with Python 2 and Python 3. The JSON API is not that 
much different to the one used by pickle: 

' ^P ov "t the JSON libvavy. 

»> import json 

»> names = ['John' , ['Johnny', 'Jack' ] , 'Michael' , ['Mike', 'Mikey', 'Mick']] 

»> names ^-- Qxtdit 3 l |S *t 

['John', ['Johnny', 'Jack'], 'Michael', ['Mike', 'Mikey', 'Mick']] 

£■- ' Trans-fovm -the Python list-o-C-lists inio a JSON list o£ lists. 


»> to_transfer = json.dumps (names) 

»> to_transfer 

'["John", ["Johnny", "Jack"], "Michael", ["Mike", "Mikey", "Mick"]]' 


TViC *fovw»3rt ,s sirftilflv*) 
tat dittevent- 


V ' pans-fov-m the JSON list ot lis+s baek 

»> from_transfer = json. loads (to_transfer) ihxo ohC P d i d 

»> from transfer 


['John', ['Johnny', 'Jack'], 'Michael', ['Mike', 'Mikey', 'Mick']] 


»> names 


['John', ['Johnny', 'Jack'], 'Michael', ['Mike', 'Mikey', 'Mick']] 


The nevj data is e*afctly the same 
as the ovi^mal list o-f hsts. 



Add a new function to the athletemodel module that, when 
called, returns the list of athlete names as a string. 

Call the new function get names from store (). 
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athletemodel function 




Err /:.r 





Solutiori 


You were to add a new function to the athletemodel module 
that, when called, returns the list of athlete names as a string. 

You were to all the new function get names from store (). 


Ae( 9et -^ e h^ OW -^ e0: . /hdt. 311 h e da h 0* Jttkk; 

, . athletes =■ aet tv-om stov-eO ^ 

£%tv-aet a list .. - . 

Jc athlete ^a^es-^ response — CathletesCeadh__athJ.Kame -for eadh__ath ‘m athletesJ 

-fvom the data. 

\retu\nr>(\respor\se) 

Retuv-n the list to the eallev. 



O 


So...rather than running 
a CQI script to create a HTML 
web page, you want me to deliver 
just the data, right? That's OK. Not 
a problem—just be sure to teli me 
which script to run... 
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With your new function written and added to the athletemodel module, create a new CGI 
script that, when called, returns the data from the get_names_f rom_store () function to 
the web requester as a JSON data stream. 

Call your new script cgi-bin/generate_names .py. 

Hint: Use applicatiori/j son as your Content-type. 
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json-generating egi script 



ExwctSe 




With your new function written and added to the athletemodel module, you were to create 
a new CGI script that, when called, returns the data from the get_names_f rom_store () 
function to the web requester as a JSON data stream. 

You were to call your new script egi-bin/generate names . py. 


po»t -fov-^et this 
"nayt Ime 

youVe 

oy\ L-wu* ov 

os ><. 



$r\ /us\r/lodal/bih/python? 


import json 
ir*po\rt athletemodel 


import yate 



p 


° youir impov-ts. 


£tavt w»th "the 
appv-opviate 
ToKte»t-*type” : 
1‘mC* 


har^CS =■ athletemodel hamCS^-fv-om^S-boV-cO 


the data -firom 
the model. 


^ pv-mtfyatestav-t_v-espohse^applidatioh/jsoh)) 
p\r’mt^sor\dunr\ps(so\rtcd^arr\es))) 


Sov-t "names”, -tKen tonvert 
■to JS OH and send -to 
STPOWT. 



Watch it! 


Take care testing 
your JSON-generating 
CGI code. 


The behavior you see 
when testing your JSON- 
generating CGI script will differ 
depending on the web browser you 
are using. For instance, Firefox might 
attempt to download the generated 
data as opposed to display it on screen. 
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If it is not already running, start your web server and be sure to set the executable bit with the 
chmod +x cgi-bin/generate_names . py command (if on Linux or Mac OS X). When you're 
ready, grab your favorite web browser and take your new CGI for a spin. 



The xelo server s 

mwrwatio*» 
£oir*tiv*w»s th3t the 
C£j I ev-etwted- 


File Edit Window Help GeneratingJSON 


$ python3 simple_httpd.py 
Starting simple httpd on port: 8080 


localhost - 
localhost - 
localhost - 
localhost - 
localhost - 
localhost - 


[18/Sep/2010 06:31:29] "GET /cgi-bin/generate_names.py HTTP/1.1" 200 - 
[18/Sep/2010 06:35:29] "GET /cgi-bin/generate_list.py HTTP/1.1" 200 - 
[18/Sep/2010 06:35:35] "POST /cgi-bin/generate_timing_data.py HTTP/1.1" 200 - 
[18/Sep/2010 06:35:38] "GET /cgi-bin/generate_list.py HTTP/1.1" 200 - 
[18/Sep/2010 06:35:40] "GET /index.html HTTP/1.1" 200 - 
[18/Sep/2010 06:35:49] "GET /cgi-bin/generate names.py HTTP/1.1" 200 - 


That worked! 

Now all you have to do is arrange for the Android emulator to request the 
data within a Python script and display the list of names on the smartphone 5 s 
screen. How hard can that be? 
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two apis 


The SL4A Android API 


The SL4A technology provides a high-level API to the low-level Android API, 
and SL4A 5 s API is documented in the online API reference: 


http://code.google. com/p/android-scripting/wiki/ApiReference 


Recall the code from earlier, which demonstrated a minimal Android SL4A 
app: 


,mport the "android” 
liWav-7 and treate a new 
3pp ohiett instante* 



Create an appropriate 
^cssagc a*d display it 



import android 

app = android.Android() 

msg = "Helio from Head First Python on Android" 
app.makeToast(msg) 


Six calls to the Android API let you create a list of selectable items in a dialog, 
together with positive and negative buttons, which are used to indicate the 
selection your user made. Note how each of the calls to the Android “dialog” 
API results in something appearing on screen. 



Always start with an import 


import android 


app = android.Android() 


Create an Android 
app objett* 


app.dialogCreateAlert("Select an athlete:") 
app.dialogSetSingleChoiceltems(['Mikey', 'Sarah', ’James', 'Julie']) 
app.dialogSetPositiveButtonText("Select") 
app.dialogSetNegativeButtonText("Quit") 
app.dialogShow() 

resp = app.dialogGetResponse().resuit 


Pisplay youv- dialog 

o* the phohe. 


Wavt «er a resfo*e 
jfv-ow * scc ’ 
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AII o-C tw.s 
) 

pvo<yrar* s 
messa^es ave m 
o«e flate- 


The name 

•f the C$| 

stvipt -fco Vuh 
oh the web 
sevvev 


This dodes a 

w\ess...dah you 

jf «* rfc? 


V 


Androict Code Magnets 


Here is the code to a program that queries your web server for the list of names as a JSON array 
and then displays the list on the smartphone.The only trouble is, the second half of the program 
is a bunch of mixed-up code magnets at the bottom of the screen. Your job is to rearrange the 
magnets to complete the program. 

import android 


import json 
import time 

from urllib import urlencode 
from urllib2 import urlopen 


t)o the usual ir*povts...these 
ohes pull ih web dlient 
fuhd-fciohality. 



hello msg 
list_title 
quit_msg 
web server 


= "Welcome to Coach Kelly's Timing App" 
= 'Here is your list of athletes:' 

= "Quitting Coach Kelly's App." 

= 'http://192.168. 1.33:8080' _ 


get names egi = '/egi-bin/generate names.py' 


def send to server(uri, post data=None): 
if post_data: 

page = urlopen(uri, urlencode(post^data)) 
else: 

page = urlopen(uri) 
return(page.read().decode("utf8")) 


Cbah$e this to the 
web addvess that s 
yuhhih^ youv web 
sevvev. 

This fuhdtioh takes both a 
web addvess (uvl) ahd so*e 
optiohal data (post_data) 
3hd sehds a web ve^uest to 
youv web sevvev. The web 



athlete names 


sorted(j son.loads(send_to_server(web_ser ver + get_names_cgi))) 

a PP-dialogGetRe 



resp = 


!S Ponse 


app.dialogShow() 


app.dialogCreateAlert(list title) |- 


how long=2) 


0-resuit 

status_update(hello 


def status__update (msg , 
app.makeToast(msg) 
time. sleep (how__long) 
app.dialogSetNegativeButtonText(’Quit') 


app. dial ogSetPositiveButtonText 



app.dialogSetSingleChoiceltems(athlete_names) 
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android query 



Android Code Magnets Solution 

Here is the code to a program that queries your web server for the list of names as a JSON array 
and then displays the list on the smartphone.The only trouble is, the second half of the program 
is a bunch of mixed-up code magnets at the bottom of the screen. Your job was to rearrange the 
magnets to complete the program. 

import android 
import json 


import time 


from urllib import urlencode 


from urllib2 import urlopen 


hello msg 
list_title 
quit_msg 
web server 
get names egi 


"Welcome to Coach Kelly's Timing App" 
'Here is your list of athletes:' 
"Quitting Coach Kelly's App." 

'http://192.168.1.33:8080' 

'/egi-bin/generate names.py' 


def send to server (uri, post data=None) : 
if post_data: 

page = urlopen (uri, urlencode(post_data)) 

Cveate Awtroid else: 

>b\ett- page = urlopen (uri) 


a?? <*>)< 


This is a litti e 


Sav 

"hello”- 


N» 


return(page.read().decode("utf8")) 


app = android.Android() | 


def status_update(msg, how_long=2): 
app.makeToast(msg) 
time.sleep(how_long) 

stat us__up date (hello msg^^ 


& .Y.. .7.. 3... . to your server, the» 

W ^ ^ ^ 
? we a >»i« W 





app.dialogCreateA lert(list_title) 

app dialogSetSingleCh oiceltems(athlete_names 

app.dialogSetPositiveButtonText('Select') 


app.dialogSetNegativeButtonText( 1 Quit') 


Create a W-tatWd 
dialo^ hom the list ot 
a-bVilc-bc naw>es- 


app.dialogShow() 


resp = app.dialogGetResponse().resuit 

£ay w bye bye- 


status_update(quit_msg) 


<..t. .fflf. use r .i? tap a button, 

theh assig h the resuit to “resp". 
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Recall that (for now) your Android Python Scripts run within the emulator, not within IDLE. So use the 
tools /adb program to copy your program to the emulator. Call your program coachapp . py. When 
the code is copied over, start SL4A on your emulator, and then tap your script's name. 



Ta\> your 
■the» tap the 

w yu» vjhccl- 


a r> 


SSS4 tlrtnd2.2 


• 2 : 


say_cha/iiy 
■3* sayjiml Ipy 
•3* say_wea\W.py 
< 3 * showdialo^N^ 
(Jl speak.py 
$ take_picture.py 


(A ©> (?l 


S5S4 dr«id2.? 


SRQG 11:42av 


A»d there they a<re...Coaeh 
Kelly s athletes. 


0 Here is your list of athletes: 


James Lee 
Julie Jones 
Mikey McManus 
Sally Sanchez 
Sarah Sweeney 

S«lect I quit 


© 0 
©@ 


1 

2 ii 

3 

4 

5j 

6 

7 1 

8 

9 

0 

Q 

W 

E 

R 

Tjj 

Y 

U 

1 

0 

P 

A 

S 

D 

F 

G 

H 

J j 

K 

L 

dii 

<*] 

❖ 

z 

X 

C 

V 

B 

N 

M 

. 



SYM 

@ 

___ 

/ 

i 



'niHE® 11:42 Ar 


This is looking really good! Your app has communicated with your web server, 
requested and received the list of athlete names, and displayed the list on your 
emulator. 



If you app doesn 5 t run, don 5 t panic. Check your code for typos. 

Run your app again in the Python terminal by tapping on the little terminal icon to the 
left of the “run wheef’ within SL4A. If your code raises an error, you’11 see any messages 
on the emulatobs screen, which should give you a good idea of what went wrong. 
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positive or negative 


Select from a list oh Awdroid 

When your user taps on a button, the “resuit” of the call to 
dialogGetResponse () is set to positive if the first button is tapped 
or negative if the second button is tapped. In your code, you can check 
the value of resp, which is a dictionary, and the which key is set to either 
positive or negative. 

A subsequent call to dialogGetSelectedl tems () returns the index 
value of the selected list item. 



« rs 


5554:droid2.2 


/hdcx i-ter* O 


Inde* rtem I 


Inde* ite** 2- 


Inde* flem 3 


Inde* ite» T 


11:42 AM 


© Here is your list of athletes: 


James Lee 


Julie Jones 


Mikey McManus 


Sally Sanchez 


Sarah Sweeney 


Select 


Quit 


The "positive" button 


MENU 


1 

2 

3 

4 

5 § 

6 

7 1 

8 

9 

0 

Q 

w 

E 

R 

T 

Y 

Ug 

I 

0 

P 

A 

S 

D 

F 

G 

H 

J 

K fl 

L 

DEL 

<Z1 

□ 

Z 

X 

C 

V 

B 

N 

M 

• 



SYM 

@ 

, _ , 

7 i 

t | 



The "negative” butto* 


So...if the positive button is tapped, you can index into the list of athlete names 
to see which athlete was selected from the displayed list. The selected name can then 
be sent to the web server to request the rest of the athlete 5 s data using the send_ 
to server () function. 


You can use this behavior in the next version of your code. 
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Assume that you have a CGI script called cgi-bin/ 
generate_data . py, which, when called, requests the data 
for a named athlete from the server. 

Provide the code (which includes a call to thensend_to_ 
server () function) to implement this functionality: 



Additionally, write the code required to display the list of times returned from the server within an Android 
dialog. 


Hints: Use the dialogSetl tems () method from the Android API to add a list of items to a dialog. Also, 
remember that the data arriving over the Internet will be formatted using JSON. 
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ask for an athlete 


^Jharpen your pencil 

Solutiori 


O 



You were to assume that you have a CGI script called cgi-bin/ 
generate_data . py, which, when called requests the data for 
a named athlete from the server. 

pvovide tbc *ame You were to provide the code (which includes a call to 

-the C£) I -b v-uw- the send_to_server () function) to implement this 

functionality: 


$et_data_d<y — V d$i-bi h/3er\evate__datapy* 
sehd_to__sevve\r(web__sevvev + get_data — £31, {'y/hidh__atblete*: \wbidb__ atblete}) 


to "bViC web sev-vev, 
tD^cbbcv >M*»-bVk tbe 
atblete *ame 



^—Ihdlude tbe data- 


O 


Additionally, you were to write the code required to display the list of times returned from the server within 
an Android dialog: 


£ 


IMidh butfcoh y/as pvessed? 


Whe* yotfr «*se<r # 'yd: .v2\.. The m de* value 

UiW 'wov-k owt —^ seled-ted_athle-te — appdialo^etSelettedltemsOYesuliCOJ ,s ( m 

•the 'mde-x- valwe --- |- is t o-f (results 

dhoscv** .. y-gtuv*hed •fvom tbc 



Look up the 
athlete s hame usih^ 
the ihdex value. 


y/hidh atblete — atblete hamesCseledted atblete 3 


Pyhamidally 
dv-eate tbe 
dialog s title 


atblete =■ jsohloads(seKd_to_sevvev(v/eb__sevvev + 3et_data__dgi 

{ ; y/bidb atblete; v/bidb atblete})) 



atblete__title — wbi£b__atblete + ; top 3 ti^es:* 




dialog 


«Sehd a hew 
web ve^uest 
to the sevvev 
to -fet^h the 
athlete ; s data. 


appdialo^CveateAlev-ttatblete^title) 


Tbe usev- heeds 
to see ohly tbe 
data tbis tiw>e, so 
you hccd to use ^ 

w dialo^etltcms() . 


appdialo^Setltemsfatbleteriop^J) 


appdialo^SetPositiveButtohTe^tt^^) 

appdialo^Shov/O 


Set tbe smjte' 
buttoh s te*t- 


I^Vait tov a tap ... ; .. ; • • 

-firom the usev-^ rcs P 1:1 a PP ^‘alo^qetRespohseO.vesult 
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The athlete'$ data Cfrl script 

Here’s the code for the cgi-bin/generate_data . py CGI script, which 
takes a web request and returas the indicated athlete’s data from the model: 


Pvofcess “the 
data Wrlh 
the ve^uest 
c^tvadt the 
athletes *ame. 


#! /usr/local/bin/python3 

import egi 
import j son 
import athletemodel 
import yate 



^et ali the data 
■firom the model. 



athletes = athletemodel.get_from_store() 
form_data = egi.FieldStorage() 

athlete__name = form_data [' which_athlete' ] . value 
print (yate. start_response (' applicatiori/j son')) 
print(json.dumps(athletes[athlete name])) 



S-bv-t a web 
respowe, wi-fch 
as ihe data type- 


The complete Android app, so far 


,ndli*de tbe 'mdif-ated 
abietes data m be web 
vesponse, tovmatted by OSOK- 


YouVe made quite a few changes to your program at this stage. Before you te st it 
on the Android emulator, take a moment to look at your code in its entirety : 


import android 
import json 
import time 

from urllib import urlencode 
from urllib2 import urlopen 

hello msg = "Welcome to Coach Kelly's Timing App" 

list title = 'Here is your list of athletes:' 
quit_msg = "Quitting Coach Kelly's App." 

web_server = 'http://192.168.1.34:8080' 

get names egi = '/egi-bin/generate names.py' 
get data egi = '/egi-bin/generate data.py' 


The v-est 7 ouy * 

dode is oy\ “the 
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app code, continued 


def send to server(uri, post data=None): 
if post_data: 

page = urlopen(url, urlencode(post_data)) 
else: 

page = urlopen(url) 
return(page.read().decode("utf8")) 

app = android.Android() 

def status_update(msg, how_long=2): 
app.makeToast(msg) 
time.sleep(how long) 

status_update(hello_msg) 

athlete names = sorted(json.loads(send to server(web server + get names egi))) 

app.dialogCreateAlert(list_title) 

app.dialogSetSingleChoiceltems(athlete names) 

app.dialogSetPositiveButtonText ( 'Select') 

app.dialogSetNegativeButtonText('Quit') 

app.dialogShow() 

resp = app.dialogGetResponse().resuit 

if resp['which'] in ('positive'): 

selected_athlete = app.dialogGetSelectedltems().resuit[0] 

which athlete = athlete names[selected athlete] 

athlete = json.loads(send to server(web server + get data egi, 

{'which athlete': which athlete})) 

athlete_title = athlete['Name'] + ' (' + athlete['DOB'] + '), top 3 times:' 

app.dialogCreateAlert(athlete_title) 

app.dialogSetltems(athlete['Top3']) 

app.dialogSetPositiveButtonText ( 'OK') 

app.dialogShow() 

resp = app.dialogGetResponse().resuit 
status_update(quit_msg) 
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// 


Let's give the latest version of your app a go. Copy the app to your emulator, and put the new CGI 
script in your cgi-bin folder on your web server (remember to set the executable bit, if needed). 
What happens when you run your latest app using the emulator's Python shell as opposed to the 
run wheel"? 


You ave dur*ped »**to 
Python shell W.th a vathev 
n&bj evvov messa^e. 







5554:droid2.2 


Tracetoack («ost recent call last): 

File "/■it/sdcard/sl4a/scrlpts/coachapp.py“, 11 

^appT^s^ogSet I tens (athlete [ ’ Top3 ’ ]) 

TypeError: qst Indices «ust be integers, not str 


MENU 




5 


DEL 


Process has exited. 
Close terminal? 


Yes 


SYM 


A-ftev veadM evvov 
messa^e, tlitk 'Yes” to «W 


Yikes! Your code has a TypeError, which is crashing your app when you try 
to display the selected athlete 5 s timing data. Why do you think this is happening? 
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debugging data 


The data appears to have chawged type 



O 



Let 5 s add a debugging line of code to your GGI script to try and determine what’s 
going on. Recall that the GGI mechanism captures any output your script sends 
to Standard output by default, so let’s use code like this to send your debugging 
messgage to the web server 5 s console, which is displaying on Standard error. 


|**povt w sys w 
-fyomr\ “the 

standavd libvav-y- 


import sys 


print(json.dumps(athletes[athlete_name]), file=sys.stderr) 


"the output 

-hroi* M pyihtO w io 

"stdevir”, vaihev- 
than -the detaJ-fc, 
v^hidh is "sidoui". 


Run your app again and, of course, it’s stili crashes with a TypeError. 
However, if you check your web server 5 s console screen, youdl see that the 
data being sent as the JSON web response is clearly visible. Notice anything? 



This is d Ii st o£ 
athlctc tiw»*mj 
values...but wheve s 
the y\a**e a^d 
P 0?> values? 


I File Edit Window Heb JustWhatsInTheData 


$ python3 simple_httpd.py 
Starting simple_httpd on port: 8080 

192.168.1.33 - - [18/Sep/2010 17:40:04] "GET /cgi-bin/generate_names.py HTTP/1.1" 200 - 
192.168.1.33 - - [18/Sep/2010 17:40:08] "POST /cgi-bin/generate_data.py HTTP/1.1" 200 - 
["2-44", "3:01", "2.44", "2.55", "2.51", "2:41", "2:41", "3:00", "2-32", "2.11", "2:26"] 
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JSON caw t hawdle your custom datatypes 

Unlike pickle, which is smart enough to pickle your custom classes, the 
JSON library that comes with Python isn’t. This means that the Standard 
library 5 s JSON library can work with Pythonh built-in types, but not with 
your AthleteList objects. 

The solution to this problem is straightforward: add a method to your 
AthleteList class to convert your data into a dictionary, and send that 
back to the app. Because JSON supports Python’s dictionary, this should work. 



Let’s create a new method in your AthleteList class. Called to_dict (), your new 
method needs to convert the class’s attribute data (name, dob, and top3) into a dictionary. Be 
sure to decorate your new method with @property, so that it appears to be a new attribute to 
users of your class. 


ilierei are no 

Diimb Questions 



What’s the purpose of this @property thing again? 


The @property decorator lets you specify that a method is to be presented to users of your class as ifit were an attribute. If you 
think about things, your to_dict () method doesn’t change the state of your objecfs data in any way: it merely exists to return the objecfs 
attribute data as a dictionary. So, although to_dict () is a method, it behaves more like an attribute, and using the @property 
decorator let’s you indicate this. Users of your class (that is, other programmers) don’t need to know that when they access the to_dict 
attribute they are in fact running a method. AII they see is a unified interface: attributes access your class’s data, while methods manipulate it. 
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data to dictionary 



Souitiort 


Let’s create a new method in your AthleteList class. Called to_dict (), your new 
method needs to convert the class’s attribute data (name, dob, and top3) into a dictionary. Be 
sure to decorate your new method with @property, so that it appears to be a new attribute to 
usersof your class. 


P edovaie Y owV ’ r? $p*-ope*"ty 

1\CW method 'Mit.n - /.i; ' '. 

"$yvoyev~ty’ • —' det as_difr(tsd£) : ^ C 3 hew “^thod. 


scl-f 

'VOty'- sel-p.dob, 
sel-p-tof^}) 

:.t.,.; 

Did you V"CrwCrwbcV" to use "sel-p* ? 


Retuvn a dittionavy o-f tbe objedts 
data attvibutcs. 



As well as updating your AthleteList 
class code, be sure to change cgi-bin/ 
generate-data . py to return a 
dictionary, rather than the object instance, 
when servicing its web request. 

While you’re making changes, adjust the 
coachapp . py app code to include the 
athlete’s name and DOB values in the 
second dialog 5 s title. 
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mobile app development 



With your changes applied to AthleteList. py, cgi-bin/generate_data . py and 
coachapp . py, use the adb tool to copy the latest version of your app to the emulator. Let's see 
how things work now. 


« r> r> 


coachapp.py - /Users/barryp/HeadFirstPython/chapter8/coachapp.py 


Heves “the Lodt 

if resp[ 1 which ’ ] in (‘positive’): 

selected athlete = app.dialogGetSelectedltems().resuit [ 0 ] 

whichathlete = athletenames ( selectedathlete ] 

athlete = json.loads(sendtoserver(web server + get_data_cgi, 

{'which athlete’: which athlete})) 

that 70UV ap\> use* 
m response “to 3 * 
a-thlc-tc scietW- 


athlete_title = athlete [' Name ’ ] + ' (’ + athlete [’ DOB ' ] + '), top 3 times:' 

app.dialogCreateAlert ( athletetitle ) 

app.dialogSetltems(athlete[ ' Top3 ' ]) 

app.dialogSetPositiveButtonText( ' OK ' ) 

app.dialogShow( ) 

resp = app.dialogGetResponse( ) .resuit 



status update(quit msg) 

1 

« n r\ _ 


55S4:droid2.2 



Ln: 55 Coi: 0 


SEI e 7:28 


0 Here is your list of athletes: 


James Lee 
Julie Jones 

Mikey McM^ujs ^ y 

Sally SanchezXbtP;- 2 

d i 

Sarah|weeney 1 p J\ 


A r> 


Quit 


^ ^ 


5554:droid2.2 


SEI fi 7:29 


AM 


© Sally Sanchez (2002-11-24), 
top 3 times: 


2.26 


Your app displays the selected 
athlete’s top three times on 
screen. How cool is that? 


2.31 


1| 

2 
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file transfer over wifi 


Rim your app oh a real phowe 

Now that your app is running successfully on your emulator, it 5 s time to try it 
on a real phone. This is where things get interesting. 

There are many options when it comes to copying your code to a real device: 

• Use file transfer over Bluetooth. 

• Use file transfer with a USB connection. 

• Use the Android SDK’s adb tool with USB. 

• Use a file transfer tool over WiFi. 

Unfortunately, which technique to use (and which work) depends very much 
on your phone. 

At Head First Labs, weVe had the greatest and most consistent success with 
the last option: use a file transfer tool over WiFi. 


y» I 

Watcli it! 


These 
instructioris 
do not work 
on the 
emulator. 


The Android emulator 
does not currently support 
Google’s Android Market, 
which you’ll need access to 
use when following along 
with the instructions on 
these pages. 


Step 1: Prepare your computer 

To transfer files securely between your Android phone and your computer, 
enable SSH file transfers by running an SSH server on your computer. How 
you do this depends on the operating system you are running: 

• Windows: download one of the many free SSH servers. 

• Mac OS X: enable remote logins. 

• Linux: install and enable OpenSSH Server. 


Step Z: Install AndFTP on your Android phone 

Use the Android Market on your phone to find and install the AndFTP app. 
This excellent tool lets you transfer hies to and from your Android phone over 
FTP, SFTP, and FTPS. 

To use it with the SSH server running on your computer, you’11 want to select 
SFTP as the file transfer protocol within the app, because AndFTP defaults 
to using the FTP protocol. 


Let’s take a look at what’s involved. 
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mobile app development 


Cowfigure AwdFTP 


With AndFTP running on your phone, configure it to connect to your 
computer ( Hostnome ) using SFTP as the transfer protocol (Type). Leave the Port, 
Username , Password , and Remote dir entries as they are, but change the Local dir 
entry to /sdcard/s!4a/scripts. 



Cba^e tWs “to 

be ibc >wcb *ar»e ov 
addvess o-f youv 
scv-vcv. 


Sei iliis io "/sddard/slfa/scripis" 

* hich enswres files iransterred y 
iironr, your server a»r e added io 

SLfA- 


Ea (Tj [Ei B A 


St 


t raHll 


03 


PM 


- 


FTP server settings 


AdvarKcd 


^ - 


yourserverxom 


Ttos: 


FTP File Transfer Prc 


opuorial [default is 21} 


POrt 

U«rname: 
Pasword; 
,ii dir 

Remote dir 


Qpfional 


optional 


/sdcard 


/yourfolder 


sjve 


Bitk 


Be suire io sei 

this io i{ £ppp J 

The value f*. 
‘PoU" should 
Change io 22. 


Be sure io ia? "Save". 


With the connection set up, tap AndFTP’s Connect button to establish a 
connection to your SSH server, entering your Username and Password when 
prompted. 

With the connection to the server established, navigate to the server folder 
containing the file(s) you want to transfer to the phone, mark the files for 
download, and tap the Download button. 

When the download completes, click Disconnect to terminate the connection 
between the phone and your computer. If you transferred a Python program, 
it should now be added to the list of Scripts within SL4A. 

It’s time to let Coach Kelly take a look. 



& b iu*iam m ch. 3 

& hella.vmrldpy 
8* nn>riYy_t, Vl » i5|JlPj . 


"zwjlme.py 
>ia / weather.p 

ih ^Wltilhs.py 
speak. rjv 


P/Ctur^, 


Y< 


IS 


ouv- app 
eady/ 
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app complete (almost) 


The coach is thrilled with his app 





VuhhihJ) Oh thc 
s phohe. 


Welcowe to the future! 

YouVe delivered a solution that automates interaction with your website while 
providing a modern interface on an Android phone. Your app allows your 
users to access web data directly on their mobile device. 



The fact that your server code runs on Python 3 and your Android client code 
runs on Python 2 makes very little difference: ii s alljust Python code\ after ali. 

All that 5 s left to do is write some code to satisfy Coach Kelly 5 s latest request, 
and you’11 get to that in the next chapter. 


This is great work. 
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mobile app development 



Your Pythow Toolbox 

You’ve got Chapter 8 under your 
belt and you’ve added some key 


Python techiques to your toolbox. 




BULLET POINTS 


■ The j son library module lets you 

convert Python’s built-in types to the text- 
based JSON data interchange format. 


■ Use j son. dumps () to create a 
stringed version of a Python type. 


■ Use j son. loads () to create a 
Python type from a JSON string. 

■ Data sent using JSON needs to 
have its Content-Type : set to 
application/j son. 


■ The urllib and urllib2 library 
modules (both available in Python 2) 
can be used to send encoded data from 
a program to a web server (using the 

urlencode () and urlopen() 

functions). 


■ The sys module provides the sys . 

stdin, sys . stdout and sys . 
stderr input streams. 
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nicinage your data 



^ Handling input 




The Web and your phone are not just great ways to display data. 

They are also great tools to for accepting input from your users. Of course, once your 
webapp accepts data, it needs to put it somewhere, and the choices you make when 
deciding what and where this “somewhere” is are often the difference between a webapp 
that’s easy to grow and extend and one that isn’t. In this chapter, you’ll extend your webapp 
to accept data from the Web (via a browser or from an Android phone), as well as look at 
and enhance your back-end data-management Services. 


this is a new chapter 
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add data anywhere 


Your athlete times app has gone National 



i ■# 


The National Under age Athletics Committee (NUAC) took one look at your 
Android app and realized it 5 s just what they need.. .almost. 

There are many ways to improve your webapp, but for now, let 5 s concentrate 
on the committee 5 s most pressing need: adding a new time value to an existing 
athlete’s data set. 

Adding new data to text files isn’t going to work: there are just too many 
coaches around the country adding data. The committee wants something 
that 5 s user friendly from any web browser or Android phone. 


Can you help? 
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manage your data 


Use a forw or dialog to accept mput 



On the Web, your user interacts with your web form and enters data. When 
she presses the submit button, the web browser gathers up all of the form’s 
data and sends it to the web server as part of the web request. 

On your Android phone, you can use the dialogGet Input () method to 
get input from the user, then mimic the behavior of the web form 5 s submit 
button in code. 

In fact, you Ve done this already: check out this line of code from your 
coachapp . py app, which sends the selected athlete name to your web 
server: 


athlete = json.loads(send_to_server(web_server + get_data_cgi , { 'which_athlete': which_athlete})) 


Heres where -the 
daia is meluded 
with -the web 
ire^ues-t. 
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form action 


Create an HTML forni template 

Let’s extend yate . py to support the creation of a HTML form. Take a look 
a this simple form, together with the HTML markup used to produce it. 



form.html 



► 

+ * http://localhost:fiOSQ/form.htrnl 

C (Qt Google 

)l 

PQ :::: 

Apple Yahoo! Coogle Maps YouTube 

Wiki pedia Populare 



Entcr a tirning value: 

( Se ™0 



The na*c of -the C$l 

sdript to send the 
form s data to. 


Clitk the "Send” 
button to swlomit the 
-form s data to youv 
web server- 



<form action= M cgi-bin/process-time.py M method= M POST M > 
Enter a timing value: 

<input type= M Text M name= M TimeValue M size=40> 

<br /> 

<input type= M Submit M value= M Send M > 

</form> 


When your user clicks on the Send button, any data in the input area is sent 
to the web server as part of the web request. 


On your web server, you can access the GGI data using the facilities provided 
by the Standard library 5 s egi module: 


t “the data sent 

-fvom the -fovi* as 
y avt o( the web 
v-e<\uest. 


import egi 

form = egi.FieldStorage() 
timing_value = form["TimeValue"].value 

— ^ 



hua , ™ ™ SB Sto to. 


r » 


The egi module converts the data associated with the web request into a 
dictionary-like object that you can then query to extract what you need. 
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Let’s turn the HTMLform from the previous page into a template within the yate .py module. 

O Start by creating a new template called templates/form.html that allows you to 
parameterize the forrrTs CGI script name, method, input tags, and submit button text: 


O 


the template ready, write the code for two functions you intend to add to yate. py. 

The first, called create_inputs (), takes a list of one of more strings and creates HTML <input> tags 
for each string, similar to the one that accepts TimeValue on the previous page. 

The second, called do_f orm (), uses the template from Part 1 of this exercise together with the create_ 

inputs () function to generate a HTMLform. ^- , r lk/r ^ 

a a list of <IHPUT> 

■fcag «ames. 


def create inputs(inputs list) 


Retuv-h the 
$c»craicd -bgs io 
ihe dallev. 


kINPuT f a * d a l,si 

y return (html_input£y__- -WTU I > tag «ar»es are veguived avgur^U 

def do form(name, the_inputs, method="POST ”, text="Submit n ): 

. 

The HTTP method a*d te*t 

{o -the u £vW*t” butto» have 
scabie de-fault values. 


Substitute the 
avauroeiftts ay\d 

^enevated <INPWT> 
tay mto the \ 
template to evcate 
the 'forw». 


return(form.substitute(cgi_name=name, http_method=method, 

list of inputs=inputs , submit text=text)) 
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html form template 



You were to turn the HTML form into a template within the yate. py module. 

RctSe O You were to start by creating a new template called templates/f orm.html that allows you 

UitiOM 


to parameterize the form’s CGI script name, method, input tags, and submit button text. 


The C$l 


<-Pov-m afrfcio»—fdAi_r>ame method—fh'bfcf__me-thod> 


The list ot <IMPIAT> 

name a»d assofciated / c . ~r. " ^ . V . r . I i , /-- "“3* and the submit 

HTTP -aw « ^. i Wj«!«. i /> w»; M i. slv> 

<input “type—value—fsubmit_tc%*t></'fo\rirw> ^ ^meteriz-ed- 


HTTP 

^av-ametev-iied 


O 


With the template ready, you were to write the code for two functions to add to yate. py. 

The first, called create_inputs (), takes a list of one of more strings and creates HTML <input> tags 
for each string, similar to the one that accepts TimeValue. 

The second, called do_f orm (), uses the template from Part 1 of this exercise together with the create_ 
inputs () function to generate a HTML form: 


def create inputs (inputs list) : 


Take caek name 
a^d dv-eate an 

<|NPWT> ta* 


TWis “dontinuation 
dhav-adtev- lets you split 
a lon^ l‘mc o£ dode ovev 
mulbplc lines. 

i 


html__inputs — ” 

-for eadh__input in inputs__list : 

htm|__inputs — htm|__inputs + Vmput type— W 7e*t” name —*** + 

eadh_input + w sizje— 


return(html_inputs) 

def do_form(name^ the_inputs, method="POST n , text= n Submit") 

^vab tke Wl ^ openrtemplates/^m.KtmlO as toirmt: 

template -fv-om to\rm te*t — tormt.readO 

youv diSK- 

inputs — d\reate_jnputs(the_jnputs) 

C««te a ' &Vm ^m P laWt*-mJ**t) . C ^ eate the l,st ^ <lmT > tys. 

template tov-m.. 

return(form.substitute(cgi_name=name, http_method=method, 

list of inputs=inputs, submit text=text)) 
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Here's the code to a CGI script called cgi-bin/test-f orm. py, which generates the HTML form 
from earlier. As you can see, there's nothing to it. 


#! /usr/local/bin/python3 

Always s-fcaU wi-th a 

Cql v-espohse. 

Py^amidally tveaie tbe &*•», 
suyylym^ a*>y ar«y»»>wts as ve<\uived- 

) 

import yate 

print(yate.start response( 

'text/html')) 

V 

print(yate.do_form('add_timing_data.py' , ['TimeValue'], text='Send')) 

— 


Set the executable bit (if required on your OS) using chmod + x test_f orm . py , and then use 
your browser to confirm that your HTML form-generating code is working. 


sterti* URL CflsArift mto 

cb bvovjscv" s lotstion b3V‘. 


youv 


* o 


rs 


http:i^calhost: 8080 /cgi-bin/test_form.py 


* 


+ 

http: //localhost:80a0/cgi-bin/te5t_form.pv C 


C ( Q, t Google 


m 


Apple Yahoo! Googlie Maps YouTube Wikipedia Popular' 


Enter a timlng value: 


Submit 


The $e»cY-alcd HTML 
appeavs wi-thin 
ihe bv-owsev-'s window. 


Use youv U-ovxsev-S 
l/ie\w Souirie" wienw 
opiioh -to don^iv-w -tha-t 
■tbe ^encraled -fov-r» is 
exad-fcly what you need- 

l 




Source of http://localhost;8080/cgi-bin/test form.py 


orm act i on= 11 cg i -b i n/adcLt i m i ng_data. py 11 method= 11 POST 11 > 
Enter a timing value: 

-dnput type= M Text" name="TimeValue" size=4@> 

<br /> 

-dnput type="Submit" value="Send"> 

Vform> 


A 


Great. You Ve extended yate . py to support the creation of a simple data 
entry form. Now all you need to do is to decide what happens once the data 
arrives on your server. 
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data delivery 


The data is delivered to your CftI script 

In addition to running your webapp, the web server also arranges to deliver 
any submitted form data to your waiting CGI script. Python’s egi library 
converts the data into a dictionary and, as you already know, provides you 
with convenient access to the submitted data: 

AII o P your -fovinr/s data 

«r as ^ addied the 

TO\rm dietiohary. 


import egi 


form = egi.FieldStorage 



Additional information about the web request is also available to you via the 
web servebs environment. Typically, you won’t need to access or use this data 
directly. However, occasionally, it can be useful to report on some of it. 

Here is some code that takes advantage of Python’s built-in support for 
querying your CGI scripbs environment using the os library, assuming the 
environment values have been set by a friendly web server. Note that the data 
in the enviroment is available to your code as a dictionary. 


pisylay “the 
cyueried data 

oy\ standard 
error. 


import os 
import time 
import sys 



Bc sure to ih^ludc the 
os libv~av~y ih your lis-t o-p 
impov-ts. 


addr = os.environ['REMOTE_ADDR'] 
host = os . environ [' REMOTE_HOST ' ] ^<^7 

method = os.environ['REQUEST METHOD'] 


1 


<$wevv -tVvree e*wivo«me*>-t vawaWes 
a«a assiy. theiv valwes to vaviables. 


cur_time = time. asctime (time. localtime ()) the £urre*t t 



me. 


print(host + ", " + addr + ", " + cur_time + ": " + method, file=sys.stderr) 



Let 5 s exploit both code snippets on this page to log the data sent from a form 
to your web server s console. When you are convinced that the data is arriving 
at your web server intact , you can extend your code to store the received data 
in your model. 

Let’s write a CGI to display your form’s data. 
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CGI Magnets 

You need a new CGI script called add_timing_data.py, 
which processes the data from a form and displays the data on 
your web server’s console screen. The CGI needs to query the 
environment, arranging to display the logged data on one line. 
The code exists, but most of it is ali over the floor. Rearrange the 
magnets to produce a working program. 


Po*t SoWti 
tbis Ime vt y ou 

ave vuvmmj oy \ 

Ma* OS >< <* 

L-muH' 



TV>eve's v-eally 
notWmJ *ew 
beve- 



T*heve s hot r»u£h o-p a 
\respohse -fov how...so 
jus-t sehd badk plaih 
“tex-t to the waitihg 
web bvowsev. 


#! /usr/local/bin/python3 

import egi 
import os 
import time 
import sys 
import yate 


print(yate.start_response(’text/plain’)) 
addr = os.environ[’REMOTE_ADDR 1 ] 
host = os.environ[’REMOTE_HOST’] 
method = os.environ[’REQUEST_METHOD’] 
cur_time = time.asctime(time.localtime()) 

print (host + ", " + addr + ", " + cur_time + ": " + method + ": ", 

end= 11 , file=sys.stderr) 



print(’OK.’) 
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add timing data 



CGI Magnets Solut ion 

You need a newCGI script called add_timing_data.py, 
which processes the data from a form and displays the data on 
your web server’s console screen. The CGI needs to query the 
environment, arranging to display the logged data on one line. 
The code exists, but most of it is all over the floor. You were to 
rearrange the magnets to produce a working program. 


#! /usr/local/bin/python3 

import egi 
import os 
import time 
import sys 
import yate 


print(yate.start_response('text/plain' ) ) 
addr = os.environ['REMOTE_ADDR'] 
host = os.environ[’REMOTE_HOST 1 ] 
method = os.environ['REQUEST_METHOD'] 
cur_time = time.asctime(time.localtime()) 

print (host + ", " + addr + ", " + cur_time + ": " + method + ", 

end=' ' , file=sys. stderr) c 

fcwuire that -this Vi»tO" 

-ruhC-tion does UOT -bke a 


form = egi.FieldStorage() 


for each form item in form.keys() 


hcwli 




print (each_form_item + '->/+ form[each_form_item] . value, 


end=' ' 


print(file=sys.stderr) 

print('OK.') 


file=sys.stderr) 

Take a newlmc oy \ s-tandavd cvv-ov" 
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Let's use your form-generating CGI script from earlier to try out add_timing_data . py. As you 
enter data in the form and press the Send button, watch what happens on the web server's console. 


£yrtev- sor*e 

da-ta rn-bo youv 

xmcV> 'forw». 




http://locaIhost:80S0/cgi-bin/test_form.py 


(k http7/localhost:fi080/cgi-bin/test_form.py C Google 



QQ 


Apple Yahoo! Google Maps YouTube Wikipedia PopularT 


Enter a timlng valuc: 2.22 

'Send \ 



http://localhost: 8Giy/cgi-bin/add_timing_data.py 

i - b i n / adcf_ti m i ng_d. Google 

YouTube Wikipedia Popular 


The web bv-owsev- 
displays a vev-y 
basie irespohse. 

aii is u or. 



| File Edit Window Help DisplayingPOSTs 


$ python3 simple_httpd.py 
Starting simple_httpd on port: 8080 

localhost - - [21/Sep/2010 17:34:54] "GET /cgi-bin/test_form.py HTTP/1.1" 200 - 
localhost - - [21/Sep/2010 17:34:54] "GET /favicon.ico HTTP/1.1" 200 - 
localhost - - [21/Sep/2010 17:35:24] "POST /cgi-bin/add_timing_data.py 
localhost, 127.0.0.1, Tue Sep 21 17:35:24 2010: POST: TimeValue->2.22 


1 . 1 " 200 - 


The web sev-vev s 
sev-eer\ 

displays the data 
that avvWed, as' 
viell as t^e *>ame 
assotiated with «t- 




That worked perfectly. The data entered into the form is delivered to your 
CGI script on the your server. Your next challenge is to provide the same user 
input experience on an Android phone. 
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android data entry 


Ask for input on your Android phone 


When you ask for user input on Android, the dialog that your user sees looks 
something like this example, which asks your user to confirm or change the 
web address and port for your server. 



The myu-t dialog has 
a trtle- 

ThevVs sorwe 

addi-tiohal desev-ip-tive 

texi, (o\r message). 


Ak 0 )( buttoh 
£ovPiv~r*s -the ehtvy 




* 


VrjfmJ 


03 


PM 


- 


etoGth chat 


(0| Which server should I use? 


Please confirm the server 
addness/name to use for your 
athlete r s timrni data: 


http://192.168.1.33:8060 


CJrKtl 


A space tov" data 

enbry, v/he« tapped, 
bving ups the "sott" 
keyboavd 


/\ "Cantel" butto» 
lets you thany youv 
nkmd- 


A single Android call creates this interface for you using the 
dialogGetlnput () method: 


title = 'Which server should I use?' 

message = "Please confirm the server address/name to use for your athlete's timing data:" 
data = 'http://192.168. 1.33:8080' 

resp = app.dialogGetlnput(title , message, data).resuit 


Pressing the Ok button sets resp to the data entered into the input area. 


Pressing the Cancel button sets resp to None, which is Python 5 s internal 
null-value. 



"The resuit ot your 
Ksev-s ihtev-actioh 
W| th the dialog is 
dssiQhed fo 




Let’s create some Android data-entry dialogs. 
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Let’s create a small Android app that interacts with your user twice. The first dialog asks the 
user to confirm the web address and port to use for the web server. Assuming your user taps 
the OK button on your dialog, a second dialog pops up to request the timing value to send to the 
server. As with the first dialog, tapping the OK button continues execution by sending the newly 
acquired timing value to the web server. Tapping Cancel at any time causes your app to exit. 

Some of the code is provided for you. Your job is to complete the program. Write the code you 
thinkyou need underthis code, and call your program get2inputsapp.py: 


import android 

from urllib import urlencode 
from urllib2 import urlopen 


Theres 

new 

here...Y ouVC 

seeift ali o-f 
*tWis dode 
be-fore- 


server title 
server msg 
timing title 
timing msg 
web server 
add time egi 


= 'Which server should I use?' 

= "Please confirm the server address/name to use for your athlete's timing data:" 
= 'Enter data' 

= 'Provide a new timing value:' 

= 'http://192.168.1.33:8080' 

= '/egi-bin/add timing data.py' 


app = android.Android() 


def send_to_server(uri, post_data=None): 
if post_data: 

page = urlopen (uri, urlencode(post_data)) 



else: 

page = urlopen(uri) 
return(page.read().decode("utf8")) 
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user interaction 



ExeRciSe 
lutioH 


You were to create a small Android app that interacts with your user twice. The first dialog asks 
the user to confirm the web address and port to use for the web server. Assuming your user taps 
the OK button on your dialog, a second dialog pops up to request the timing value to send to the 
server. As with the first dialog, tapping the OK button continues execution by sending the newly 
acquired timing value to the web server. Tapping Cancel at any time causes your app to exit. 

Some of the code was provided for you. Your job was to complete the program by writing the 
code you think you need under this code and call your program get2inputsapp .py. 


import android 

from urllib import urlencode 
from urllib2 import urlopen 


server title 
server msg 
timing title 
timing msg 
web server 
add_time_cgi 


'Which server should I use?' 

"Please confirm the server address/name to use for your athlete's timing data:" 
'Enter data' 

'Provide a new timing value:' 

'http://192.168.1.33:8080' 

'/cgi-bin/add_timing_data.py' 


app = android.Android() 


def send_to_server(uri, post_data=None): 
if post_data: 

page = urlopen (uri, urlencode(post_data)) 
else: 

page = urlopen(uri) 
return(page.read() .decode("utf8" ) ) 


The -first dialog asks your 
wer to Coh-fiv» the web 
address a«d fori to us e . 



resp — appdialog^et|r>put(server_title, server_msg, web_server).resuit 


. your user M NOT tap 

it "« f i. ** H ."* ^ **** UW -tt. «tad ^ U 

v/eb__se\rve\r — \resp 3 hC>w v ^ uc - 

i-P \resp is y\o ’t <<_ _ i-f youv- uSCV " d’*d 

. . . tap o» tbe Ca^dcl butW.. ...-the app s Ch ds -the 

^ ^ S P data io wc b serv*-. 

scr\d_to__sc\rvc\r(y/cb_sc\rvc\r + add^ti^c^dji, {'Tirnih^/alue*: r\eY/__tir*e}) 
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Let's copy get2inputsapp . py to the emulator using the adb tool: 



The get2inputsapp .py app appears on the list of Scripts within SL4A. Go ahead and give it a tap: 


A rs 


SSS4 drt>id2.2 



Cancel 


oy\ -the mfu-t 

avca, A*dvo»ds 
w saf*t w keyboav-d 

uf- 


^ Which server should I use? 


Please confirm the server 
address/name to use for your 
athlete's timing data: 


Yowv nex app s ta v "t s ' 
and yo w tan ed't 
■the xeb server 
address and port- 


http://192.168.1.33:808(1 


25 


ni 


Enter a nex 
timing valwe, 
and then tap 

"0k". 


A ^ 

A 


SSS4 drc>id2.2 


(▼) Enter data 

Provide a new timing value: 


MtNU 


1 

2 

3 | 

4 

si 

6 

7 | 

8 

9 

0 

q ii 

W 

E 

R 

T U 

y y 

U 

I 

O 

P 

A 

S 

D 

F i 

G 

H 

J 

K 

L 

DII 

<2 

❖ 

z 

X 

C 

V 

B 

N 

M 


4 -» 


SYM 

@ 

, _ , 

/ 

# 



$ python3 simple_httpd.py 
Starting simple_httpd on port: 8080 

localhost - - [21/Sep/2010 17:34:54] "GET /cgi-bin/test_form.py HTTP/1.1" 200 - 
localhost - - [21/Sep/2010 17:34:54] "GET /favicon.ico HTTP/1.1" 200 - 
localhost - - [21/Sep/2010 17:35:24] "POST /cgi-bin/add_timing_data.py HTTP/1.1' 
localhost, 127.0.0.1, Tue Sep 21 17:35:24 2010: POST: TimeValue->2.22 
192.168.1.33 - - [21/Sep/2010 20:50:30] "POST /cgi-bin/add_timing_data.py HTTP/: 
localhost, 192.168.1.33, Tue Sep 21 20:50:30 2010: POST: TimingValue->2:56 


i 

200 - 

The web setrvev-^s 

log ^oh-PitrrhS 

tbe data w as 

1" 200 - 

scKrt -(Vor* youv* 


_ j 



Perfect. That 5 s working, too. Regardless of where your data originates—on 
the Web or a phone—your app can send it to your web server. 
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update which dataset? 


It'$ time to update your server data 



Yikes! I think theres a problem here... 
your server data is in two places: wi+hin 
your pickle and in the NUACs text files. 
The question is: which one do you update? 


Which of your two datasets should you update? 

If you update the pickle, the next time the put_to_store () 
function runs, your most recent update will vanish as put_ 
to_store () recreates the pickle from the data in the text files. 
Thaf s not good. 

If you update the appropriate athlete’s text file, the data in the 
pickle will be stale until put_to_store () runs again. If 
another process calls the get_f rom_store () function in the 
meantime, the update to the pickle might not have been applied 
and will appear to be missing for anyone reading your data. Thaf s 
not good, either. 


Oh, look, how lovely: I 
have a new timing value to add to 
the System. Who's going f irst? 



Web 

Server 


l 


„1 




Y<»*y texi -file YJ' 








cnrtotnly 3:3 not" 


youv 'pitkle -f ile 
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Avoid race cowditiows 




Yes, that’s one possible solutiori, but it’s a poor one. 

You might think it highly unlikely.. .but it is possible for another 
process to call the get_f rom_store () function between the text file 
update and the pickle recreation, resulting in a short period of data 
inconsistency. These types of situations are known as race conditions 
and are hard to debug when they occur. 

It’s best to keep them from ever happening if you can. 

The basic problem here is that you have one update with one piece of data 
that results in two file interactions. If nothing else, that’s just wasteful. 



0 



Yowr up—to— 
da-fce iex-fc -Pile 



''tl Youv WfoTavilY 

mdonsistea ?itkle +ilc 
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avoid race conditions 


You need a better data storage wechawisw 

Your initial text files and pickle design is fine when only one user is accessing 
the data. However, now that more than one person can access the data at any 
time, and from anywhere, your design is in need of improvement. Above ali, 
you need to avoid that race condition. 



YouV mdoyvsis-teirrt 

a*d uyset 


Listen, bud, i+'s not my fault...until 
someone, somewhere runs the "put_to_store()" 
f unction without someone, somewhere else running the 
"get_from_store()" f unction, you'11 have to do without that 
data update. I'm not a miracle worker...I just do what 
I'm told. 


O 



tlierei are no 

Dumb Questions 


x,- Surely you should have thought about this problem long 
ago and designed this “properly” from the start? 

That’s certainly one way to look at things, and hindsight is 
always a wonderful thing! However, programs have a tendency 
to start out small, then grow to provide more features, which can 
introduce complexity. Recall that the coach’s app started life as a 
simple “standalone” text-based program, which was then moved 
to the Web to support multiple users. Part of the app was then 
redeveloped for use on an Android phone. And yes, if we’d known ali 
of this ahead of time, we might have been designed it differently. 



So l’m facing a rewrite of large chunks of my code? 


Let’s see. You did build your program using the MVC pattern, 
and you are using Python, so those two facts should take the sting 
out of any potential rewrite, assuming a rewrite is what’s required 
here. 
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WouIdrTt it be dreamy if I could put my 
data in only one place and support all my 
app's requirements? But I know it's just a 
fantasy... 
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which database management system? 


Use a database management system 

You need to move away from your text file and pickle combination and use a 
real database management system. You have plenty of choices here... 




If you want rock- 
solid without the 
corporate bloat, it has 
to be PostgreSQL. 

O 


I really like 
MySQL and 
MariaDB. 


All of these fine technologies will work, but they are overkill for your app 5 s 
data requirements. And besides some of these are way beyond the NUACTs 
budget, let alone their ability to set up, run, and maintain such a system. 

What you need is something thafis effectively hidden from the NUAG yet lets 
you take advantage of what a database management system has to offer. 

If only such a technology existed... 
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Python includes SQLite 

Python 3 comes preinstalled with Release 3 of SQLite, a full-featured, zero- 
config, SQL-based data management system. 

To use SQLite, simply import the sqlite3 library and use Python 5 s 
standardized database API to program it.There 5 s really nothing to it: no 
database setup, no config, and no ongoing maintenance. 

With your data stored in SQLite, rewrite your webapp 5 s model code to use 
SQL to access, manipulate, and query your data. You can plan to move 
to one of the bigger database Systems if and when your application needs 
dictate such a move. 

SQLite sounds perfect for the NUAC 5 s data, doesn’t it? 





The material in this chapter assumes 
you are comfortable with SQL 
database technology. If you are 
new to SQL (or just need a quick 
refresher), take a look at Heod 
First SQL, which comes highly 
recommended. 


CNoie W MarketH /W.lable aw/*A*« 
500 d books ave sold and b> aw/one ww a 
valui cved^ cav"d-3 
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database connection process 


Exploit Pythonis database API 

The Python Database API provides a Standard mechanism for 
programming a wide variery of database management Systems, including 
SQLite. The process you follow in your code is the same regardless of which 
back-end database you’re using. 




Connect 

Establish a connection to your 
chosen database back end. 




Create 


Create a cursor to communicate through the 
connecton to your data. 



Interact 


Using the cursor, manipulate your 
^ data using 




Commit 



Teli your connection to apply 
all of your SQL manipulations 
to your data and make them 
permanent. 


\ 1 / 

'pooth 

' I \ 


Rollback 

Teli your connection to abort your 
SQL manipulations, returning your 
data to the state it was in before your 
interactions started. 




Close 


Destroy the connection to the 
database back end. 



Me* you fclose youv 
do**cti’*o*> youv duvsov 
is destvoyed, *too- 
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The database API as Python code 

Here 5 s how to implement an interaction with a database using the sqlite3 
module: 


This disk -Pile is used io hold 
the database ahd its tables. 



/\s aUays, 

the libvavy y°<* 
need- 


Establish a £orme£*tion 
*to a database. 



import sqlite3 


connection = sqlite3.connect(’test.sqlite’) 


cursor = connection.cursor() 


cursor. execute ( 1111 " SELECT DATE (’ NOW f ) " " ") 


Cresie a du\rsoV- to 
the data. 

Execute some £QL. 

Commit any eba^jes, _w 

makmg ther» permanent ^connection. coiranit () 

Closc your to^tcho^V) connection. close () 
wheh you Ve ( ihished. ^ 





Depending on what happens during the Interact phase of the process, you 
either make any changes to your data permanent (commit) or decide to 
abort your changes (rollback). 

You can include code like this in your program. It is also possible to interact 
with you SQLite data from within IDLEs shell. Whichever option you choose, 
you are interacting with your database using Python. 

It’s great that you can use a database to hold your data. But what schema 
should you use? Should you use one table, or do you need more? What data 
items go where? How will you design your database? 

Let’s start working on the answers to these questions. 
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design your database 


A little database design goes a long way 

Let’s consider how the NUACTs data is currently stored within your pickle. 

Each athlete 5 s data is an AthleteList object instance, which is associated 
with the athlete’s name in a dictionary. The entire dictionary is pickled. 



r r 

mnlcteLisis wi-fchih it 



Each AthleteList has the following attributes: 



With this arrangement, it is pretty obvious which name, date of birth, and list 
of times is associated with which individual athlete. But how do you model 
these relationships within a SQL-compliant database system like SQLite ? 

You need to define your schema and create some tables. 


316 Chapter 9 
























manage your data 


Pefiwe your database schema 

Here is a suggested SQL schema for the NUACfs data. The database is called 
coachdata . sqlite, and it has two related tables. 

The first table, called athletes, contains rows of data with a unique 
ID value, the athlete’s name, and a date-of-birth. The second table, called 
timing_data, contains rows of data with an athlete’s unique ID and the 
actual time value. 



CREATE TABLE timing_data ( 

— athlete_id INTEGER NOT NULL, 

value TEXT NOT NULL, 

FOREIGN KEY (athlete_id) REFERENCES athletes) 

i- 

C^s/o-te hovj tWis sthema "l'mb W 
-tables us'm^ a -foreiy* 

There can be one and only one row of data for each athlete in the athletes 
table. For each athlete, the value of id is guaranteed to be unique , which 
ensures that two (or more) athletes with the same name are kept separate 
within the system, because that have different ID values. 

Within the timing data table, each athlete can have any number of time 
values associated with their unique athlete_id, with an individual row of 
data for each recorded time. 


Let’s look at some sample data. 
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athletes and values 


What docs the data look like? 

If the two tables were created and then populated with the data from the 
NUACfs text files, the data in the tables might look something like this. 



name 

James Lee 
Sarah Sweeney 
Vera Vi 
Julie Jones 
Sally Sanchez 
MiKey McManus 


Lu is ' Mh L a f the dah ih the 

table pht look like, with 0 * e v-ow o-f 
data W eath athlete. 


)) 


dob 

2002 - 03-14 

2002 - 06-17 

2002 - 12-25 
2002 - 08-17 
2002-11 -24 
2002 - 02-24 


If you create these two tables then arrange for your data to be inserted into 
them, the NUAG’s data would be in a format that should make it easier to 
work with. 

Looking at the tables, it is easy to see how to add a new timing value for an 
athlete. Simply add another row of data to the timing_data table. 

Need to add an athlete? Add a row of data to the athletes table. 

Want to know the fastest time? Extract the smallest value from the 
timing data table 5 s value column? 

Let’s create and populate these database tables. 


Tbeves move data in tbis 
table tban sbown beve- 
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I 


SQLite Magnets 


Let's create a small Python program that creates the coachdata . 
sqlite database with the empty athletes and timing data 
tables. Call your program createDBtables . py.The code you 
need is almost ready. Rearrange the magnets at the bottom of the 
page to complete it. 


import sqlite3 


cursor.execute("""CREATE TABLE athletes ( 


athlete_id INTEGER NOT NULL, 
value TEXT NOT NULL, 

FOREIGN KEY (athlete id) REFERENCES athletes)""") 


connection.commit() 
connection.close() 
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create database tables 



SQLite Magnets Solut ion 

Your job was to create a small Python program that creates the 
coachdata . sqlite database with the empty athletes 
and timing_data tables. You were to call your program 
createDBtables . py.The code you needed was almost ready, 
and you were to rearrange the magnets at the bottom of the page to 
complete it. 


import sqlite3 



connection = sqlite3 . connect (’ coa.chd.ata. . sqlite ) 




cursor.execute("""CREATE TABLE athletes ( 




athlete_id INTEGER NOT NULL, 
value TEXT NOT NULL, 

FOREIGN KEY (athlete id) REFERENCES athletes)""") 



connection.commit() 
connection.close() 


The conr.n.i-t is*'t always re^ived with 

database syste.s, but it 
is with SQLile. 
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Transfer the data f row your pickle to SftLite 


As well as writing the code to create the tables that you need, you also need 
to arrange to transfer the data from your existing model (your text files and 
pickle combination) to your new database model. Let 5 s write some code to do 
that, too. 

You can add data to an existing table with the SQL INSERT statement. 
Assuming you have data in variables called name and dob, use code like this to 
add a new row of data to the athletes table: 



The data i» these vaviabta 
is subs-ti-tu-ted i» plaee o-f 
the ? plaeeholders. 


cursor.exeeute("INSERT INTO athletes (name, dob) VALUES (?, ?)",(name, dob)) 


t ~ ~ ««i ««7 “R? a 

^ L t0 U», loetause S^Liie f^v.des o«e W 




Reaps Batce 




Here’s a program, called initDBathletes .py, which takes 
your athlete data from your existing model and loads it into your 
newly created SQLite database. 


£ ; 1 import sqlite3 

-to the new 

database. ^ conne cti° n = sqlite3.connect('coachdata.sqlite') 

cursor = connection.cursor() 

^ab the 
data fVor* 
the existi hg 
""odei. \ 

> 


6jet the athlete j 

^a^e and POB 
-fvow\ the piekled 

data. 

Wse the IW£BRT 

statemeht to add 
a hew irow to the 
“athletes” table. 



import glob 
import athletemodel 

data_files = glob.glob/data/*.txt") 
athletes = athletemodel.put_to_store(data files) 

for each ath in athletes: 


name = athletes[each_ath].name 
dob = athletes[each_ath].dob 

'cursor.exeeute("INSERT INTO athletes (name, dob) VALUES (?, ?)", (name, dob)) 
connection.commit() 


Wake the changet) permanent 


connection.close () 
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names and numbers 


What IP is assigwed to which athlete? 

You need to query the data in your database table to work out which ID value 
is automatically assigned to an athlete. 

With SQL, the SELECT statement is the query king. Here’s a small snippet of 
code to show you how to use it with Python, assuming the name and dob 
variables have values: 


cursor.execute("SELECT id from athletes WHERE name=? AND dob=?", (name, dob)) 
-■— --——— - ——- 


Agam, the plaeeholdew mdieate 
*heire the data values ave 
substituted mto the guev-y. 


If the query succeeds and returns data, it gets added to your cursor. You can 
call a number of methods on your cursor to access the results: 

• cursor . fetchone () returns the next row of data. ^ 

• cursor . fetchmany () returns multiple rows of data. 

• cursor . fetchall () returns all of the data. 



£ath ot these twsov 
methods v-etwvir» a list_ 

ot V-OVIS. 
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Insert your timing data 

You’re on a roll, so let 5 s keep coding for now and produce the code to take 
an athlete 5 s timing values out of the pickle and add them to your database. 
Specifically, you’11 want to arrange to add a new row of data to the 
timing_data table for each time value that is associated with each athlete 
in your pickle. 

Those friendly coders over at the Head First Code Review Team have just 
announced theyVe added a clean_data attribute to your AthleteList 
class. When you access clean_data, you get back a list of timing values 
that are sanitized, sorted, and free from duplicates.The Head First Code 
Review Team has excellent timing; that attribute should come in handy with 
your current coding efforts. 


Grab your pencil and write the lines of code needed to query the 
athletes table for an athlete's name and DOB, assigning the 
resuit to a variable called the_current_id. Write another 
query to extract the athlete's times from the pickle and add them 
to the timing_data table. 

assi<\*edi b> 

d. 


%J>arpen your pencil 



Vt's 0£ ko assuge m wr 
tode -t^a-t tV>e 'Vame a»d dob 
Rabies d y\A havc values 
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database queries 


r /T,r: 

Solution 


^wev-v -bVie "athletes" 

table -f°v the IP- 



You were to grab your pencil and write the lines of code needed 
to query the athletes table for an athlete's name and DOB, 
assigning the resuit to a variable called the_current_id. You 
were then to write another query to extract the athlete's times 
from the pickle and add them to the timing data table. 


euv-so\re%edute( w ££L£CT id -fv-om athletes 1/VttERE name—? AMV dob—?”, 



ft otteh makes sense 
io split youv- exeeute 
statemeht ovem multiple 


mes. 


Rcme^ber: ^r»e, dob)) 

w *fet£hov\eO 

vetuv^s a list- 

the__du\r\rent_jd — dursor-Pet^honeOCOJ 

Take eaeh of . . ,Tr ‘. 

the w elea^ +or eadh_tir*e m athletesCeadh__athJ.dleary_data : 

l ^cthev £ursor.exe£ute(”INSERT II^TO ti^m^__data (athlete_jd, value) W\LUES (?, ?) w 

Y/ith the IP» (the du\r\reht id, eadh time)) 

withm the . — . — . — . 

‘ IN f :r ^ “ «x a> ^ «. 

statement- ^ 3(^5, »aUe the value to the "ti*>iha 

dohuettiow^ommitfv C. ^^a^nc(s) ^evmanent- data table- 



Add the code to your init DB athletes. py code from earlier, 
just after the connection . commit () call. Rename your 
program initDBtables . py, now that both the athletes 
and timing_data tables are populated with data by a single 
program. 


That’s enough coding (for now). Let’s transfer your pickled data. 
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You've got two programs to run now: createDBtables . py creates an empty database, defining 
the two tables, and initDBtables . py extracts the data from your pickle and populates the tables. 
Rather than running these programs within IDLE, let's use the Python command-line tool instead. 


|jf vou av-c iVmdows, 

y-eflate "pytVi onl' virth 



Be dave-ful "" ^ 

"to v-uh bo-fch 

programs OH L/ 

ok£c. 


| File Edit Window Help PopulateTheTables 

i 

$ python3 createDBtables.py 
$ python3 initDBtables.py 
$ 






Helio? Some+hing happened there, 
didn't it? I ran the programs but nothing 
appeared on screen...how do I know if 
anything worked? 
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sqlite manager 


SQlite data management tools 

When it comes to checking if your manipulations of the data in your 
database worked, you have a number of options: 

© Write more code to check that the database is in the 
state that you expect it. 

Which can certainly work, but is error-prone, tedious, and way too 
much work. 


Ci+e v-eally is 
too shovt. 


© Use the suppiied "sqlite3" command-line tool. 

Simply type sqlite3 within a terminal window to enter the SQLite ^ 

“shell.” To find out which commands are available to you, type . help ^ ^ w V\Mp 

and start reading. The tool is a little basic (and cryptic), but it works. 


ThaVs a ywod, 
allowed 
ovd w hclf 


e 


TWis is what 
£QLi*te Ma^ay^r 
looks like- 


Use a graphical database browser. 

There are lots of these; just Google “sqlite database browser” for 
more choices than you have time to review. Our favorite is the SQLite 
Manager, which installs into the Firefox web browser as an extension. 


\ 


<*- 


l/Vorks gvea-fc, 
but oh ly 
Filre-fox. 


oh 




SQLite Manager - /Users/barryp/HeadFirstPython/chapter9/coachdata.sqlite 


0 


Pl & f(*) 

sr m in ut 

Directory ► 

(Select Profile Database) 

a 'l 

▼ 1 


Co 


coachdata. sqlite 


A 

▼ 


Structure Browse & Search Execute SQL DB Settings 


► Master Table (1) 

▼ Tables (3) 





TABLE athletes 

Search J (_ Show AII J (_ Add j ( Duplicate J f Edit 

athletes 

4 

id 

name 

dob 

► sqlite_sequence 



1 

James Lee 

2002-3-14 

► timing_data 



2 

Sarah Sweeney 

2002-6-17 

► Views (0) 



3 

Vera Vi 

2002-12-25 

► Indexes (1) 



4 

Julie Jones 

2002-8-17 

► Triggers (0) 



5 

Sally Sanchez 

2002-11-24 


4 

6 

Mikey McManus 

2002-2-24 


c: 


SQLite 3.6.22 Cecko 1.9.2.10 


VJ. 


Exclusivc Number of files in selected directory: 10 



r 


▲ 

T 


◄ ► 


ET: 1 


al ' ^ the aihle ±“ «ne m the "athletes" Ubie. 


But how do you integrate your new database into your webapp? 
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Integrate SftLite with your existing webapp 



Joe: This should be easy. We just have to rewrite the code in 
athletemodel . py to use the database, while keeping the API the 
same. 

Frank: What do you me an by keeping the API the same? 

Joe: Well...take the get_f rom_store () function, for instance. It 
returns an AthleteList dictionary, so we need to make sure that 
when we update get_f rom_store () to use our database that it 
continues to return a dictionary, just as it’s always done. 

Frank: Ah, now I get it: we can query the database, grab all the data, 
turn it into a big dictionary containing all of our AthleteList 
objects and then return that to the caller, right? 

Joe: Yes, exactly! And the best of it is that the calling code doesn’t need 
to change at all. Don’t you just love the beauty of MVG? 

Frank: Ummm...I guess so. 

Jim: [cough, cough] 

Frank: What’s up, Jim? 

Jim: Are you guys crazy? 

Joe & Frank: What?!? 

Jim: You are bending over backward to maintain compatibility with an 
API that exists only because of the way your data model was initially 
designed. Now that you Ve reimplemented how your data is stored in 
your model, you need to consider if you need to change your API, too. 

Joe & Frank: Change our API? Are you crazy?!? 

Jim: No, not crazy, just pragmatic. If we can simplify the API by 
redesigning it to better fit with our database, then we should. 

Joe: OK, but we haven’t got all day, y’know. 

Jim: Don’t worry: it’11 be worth the effort. 


you are here ► 
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get out of a pickle 




Let's spend some time amending your model code to use your SQLite database as opposed 
to your pickle. Start with the code to your athletemodel. py module. Take a pencil and 
strike out the lines of code you no longer need. 


import pickle 


from athletelist import AthleteList 


def get_coach_data(filename): 
try: 

with open(filename) as f: 

data = f.readline() 
templ = data.strip().split( 1 ,’) 

return(AthleteList(templ.pop(0), templ.pop(0), templ)) 
except IOError as ioerr: 

print('File error (get_coach_data): ’ + str(ioerr)) 

return (None) 


def put_to_store(files_list): 
all_athletes = {} 
for each_file in files_list: 

ath = get_coach_data(each_file) 
all_athletes[ath.name] = ath 

try: 

with open(’athletes.pickle’ , ’ wb’) as athf: 

pickle.dump(all_athletes, athf) 
except IOError as ioerr: 

print('File error (put_and_store): ' + str(ioerr)) 

return (all athletes) 
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def get_from_store(): 
all_athletes = {} 
try: 

with open('athletes.pickle', ' rb' ) as athf: 

all_athletes = pickle.load(athf) 
except IOError as ioerr: 

print('File error (get_from_store): ' + str(ioerr)) 

return(all_athletes) 

def get_names_from_store(): 

athletes = get_from_store() 

response = [athletes[each_ath].name for each_ath in athletes] 
return(response) 
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out of a pickle 



SoLwtiOH 


Let's spend some time amending your model code to use your SQLite database as opposed 
to your pickle. Start with the code to your athletemodel. py module. You were to take a 
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get name3 from atorc() i 
* a t- hlctcj ~ ge ^ t_from_atoro ( ) 

, rorponoo - [athlcteo[cach_ath].namc ror eacli_dLli in dllilelej-} 

rphirn (rpgpnnqoJ _____ 


o 



This might seem a little 
drastic...but sometimes a 
redesign requires you to throw 
away obsolete code. 
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get names from store 


You stili need the list of names 

Throwing away all of your “old” model code makes sense, but you stili need 
to generate a list of names from the model. Your decision to use SQLite is 
about to pay off: all you need is a simple SQL SELECT statement. 


Reapy Batte 



Corneti io {-he 

daiabase. 


£*tv-aet the 
daia you *eed 


Fov-mulate a 
trcspohsc. 



Here’s the code for your new get_names f rom_store () 
function: 



import sqlite3 

db_name = ’coachdata.sqlite’ 

def get_names_from store(): 

connection = sqlite3.connect(db_name) 
cursor = connection.cursor () 

results = cursor.execute ("""SELECT name FROM athletes""") 
response = [row[0] for row in results.fetchall()] 
connection.close () 
return (response) 


Re-U™ the list <£ 

naw*es to the taWcr- 
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&et ah athlete $ details based on 1P 

In addition to the list of names, you need to be able to extract an athlete 5 s 
details from the athletes table based on ID. 




Reapy Bane 




Here’s the code for another new function called 

get athlete from id(): 



-the data 

assod\atcd wth 

a s^cdi-fiC- IP- 

"the W har*e” 

a^d w p0B” valucs^-^ 
-fVor» -the aihlcie 
table. 


£jet the I»st o-f 

times £v-om the 
u ~bWm^ data 
table. 


def get_athlete_from_id (athlete id): 

connection = sqlite3.connect(db name) 
cursor = connection.cursor () 



Woie the use ot the placeholdev- 
to indicate where the "athlete 

f t is inserted ini ~ 

the S#L SBLBCT ^uevy. 

results = cursor.exeeute ( MMM SELECT name, dob FROM athletes WHERE id=? MMM , 

(athlete id,)) 

(name, dob) = results.fetehone() 



Retuv-h the 
athlete^s data 

to the valler. 


results = cursor.exeeute("""SELECT value FROM timing_data WHERE athlete id=«*!"", 

(athlete id,)) 

data = [row[0] for row in results.fetchall()] 



response = { 

'Name': 

name, 


'DOB': 

dob, 


'data': 

data, 


'top3': 

data [0:3] } 


connection.close() 
return(response) 



"fake the daia bom both 

'pery results and turn it into 
a dtetionavy. 


This function is a more involved than get_names f rom_store (), but 
not by much. It stili follows the API used with working with data stored in 
SQLite. This is coming along. nicely. 

With the model code converted, you can revisit your CGI Scripts to use your 
new model API. 


Let’s see what’s involved with converting the CGIs. 
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use ids internally 



Isn't there a problem here? The 
"get_names_from_store()" f unet ion returns a list 
of names, while the "get_athlete_from_id()" function 
expects to be provided with an ID. But how does the 
web browser or the phone know which ID to use when 
all it has to work with are the athletes' names? 


That’s a good point: which ID do you use? 

Your current GGIs all operate on the athlete name, not 
the ID. In order to ensure each athlete is unique, you 
designed your database schema to include a unique ID 
that allows for your system to properly identify two (or 
more) athletes with the same name , but at the moment, 
your model code doesn’t provide the ID value to either 
your web browser or your phone. 

One solution to this problem is to ensure that the athlete 
names are displayed to the user within the view, while the 
IDs are used internally by your system to unique identify 
a specific athlete. For this to work, you need to change 
get names from store() . 
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Here is the current code for your get_names_f rom_store () function. Rather than 
amending this code, create a new function, called get_namesID_f rom_store (), 
based on this code but including the ID values as well as the athlete names in its response. 
Write your new function in the space provided. 


import sqlite3 

db_name = 'coachdata.sqlite' 

def get_names_from_store(): 

connection = sqlite3.connect(db_name) 
cursor = connection.cursor() 

results = cursor.execute("""SELECT name FROM athletes""") 
response = [row[0] for row in results.fetchall()] 
connection.close() 
return(response) 


you are here ► 
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get name’s id 



§oU#tiort 


Here is your current code for your get_names_from_store () function. Rather than 
amending this code, you were to create a new function, called get_namesID_f rom_ 
store (), based on this code but including the ID values as well as the athlete names in its 
response. You were to write your new function in the space provided. 


import sqlite3 

db_name = 'coachdata.sqlite' 

def get_names_from_store(): 

connection = sqlite3.connect(db_name) 
cursor = connection.cursor() 

results = cursor.execute("""SELECT name FROM athletes""") 
response = [row[0] for row in results.fetchall()] 
connection.close() 
return(response) 


dc-f hamCsIP^-fvom^S-toV-cO: 

dormedtio* =■ s^lite3dormedt(db_jr\arne) 
dursor — dormed*tior\du\rso\rO 



Arrahje -fc o ihdlude -the 
valuc o-P V' ih the 

SQL "£BLBCT” ^ery 


v-esults — duv-so\r.e*edute( www ££LECT *ar*e, id FR0M atbletes^O 


response =■ results.-PetdhallO 


dormedtiondloseO 




\retu\rn(\response) 


use VeW Julh^UIOr. Pt, ° h ^ ^ 


Theves no nccA to pvotess 
Vsults" m aw/ «av-assi^ 

evevytWm^ vetuvned W the 
ey*evy t° 'Ves^onse • 
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Sharpen your pencil 

This is -the “ywevatejist f/ 

C^l sdript 

#! /usr/local/bin/python3 

import glob 
import athletemodel 
import yate 


Part 1 : With your model code ready, let's revisit each of your 
CGI Scripts to change them to support your new model. At the 
moment, all of your code assumes that a list of athlete names or an 
AthleteList is returned from your model. Grab your pencil and 
amend each CGI to work with athlete IDs where necessary. 


data_files = glob.glob("data/*.txt") 

athletes = athletemodel.put_to_store(data_files) 


print(yate.start response()) 


(kzr 


W°ic the chanae to 
the title- 




print(yate.include_header("NUAC's List of Athletes")) 
print(yate.start_form("generate_timing_data.py")) 

print(yate.para("Select an athlete from the list to work with:")) 
for each_athlete in sorted(athletes): 

print(yate.radio_button("which_athlete" , athletes[each_athlete].name)) 
print(yate.end_form("Select")) 

print(yate.include footer({"Home": "/index.html"})) 


This is "o j enev-aie__l i "' m ?U^ a '^ ^ ’ 



#! /usr/local/bin/python3 


import egi 

import athletemodel 

import yate 


This Sharpen" is dontinued 
0h ™e »cxi p ane, but ho 
peekmg/ Doni +lip over uniil 
you ve amended the dode on 
this page- 


athletes = athletemodel.get_from_store() 
form_data = egi.FieldStorage() 

athlete_name = form_data['which_athlete'].value 

_ /Wther t>t'e tV ' a ^ e ' 

print(yate.start_response()) kf 

print(yate.include_header("NUAC's Timing Data")) 

print(yate.header("Athlete: " + athlete_name + ", DOB: " + athletes[athlete_name].dob + ".")) 
print (yate.para("The top times for this athlete are:")) 
print(yate .u_list (athletes[athlete_name].top3)) 

print (yate.para("The entire set of timing data is: " + str(athletes[athlete_name] .clean_data) + 

" (duplicates removed).")) 

print(yate.include_footer({"Home": "/index.html", "Select another athlete": "generate_list.py"})) 
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not done yet 



This is ita m 

"(\e»>eva-te_»>ames.fY 

C(\ V 

#! /usr/local/bin/python3 

import json 


Part 2: You're not done with that pencil just yet! In addition to 
amending the code to the CGIs that support your web browser's 
Ul, you also need to change the CGIs that provide your webapp 
data to your Android app. Amend these CGIs, too. 


import athletemodel 
import yate 


names = athletemodel.get_names_from_store() 

print (yate . start_response ( ' applicatiori/j son ' ) ) 
print (json.dumps(sorted(names))) 


/\y\d ^CV"C is “tViC ^ 

V»ev-ale_daiap/' 

C£j I- 

#! /usr/local/bin/python3 




import egi 
import json 
import sys 


import athletemodel 
import yate 


athletes = athletemodel.get_from_store() 
form_data = egi.FieldStorage() 

athlete_name = form_data['which_athlete'].value 

print(yate.start_response('application/j son')) 
print(json.dumps(athletes[athlete_name].as_dict)) 
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Sharpen your pencil 

Solutiori 

This is the "ge»ev-atejist?y 
C^l sdripi 

#! /usr/local/bin/python3 

- 

import. athletemodel 
import yate 

m * - *gi bu . • glub ( >^ du 

athletes = athletemodel. 


Part 1 : With your model code ready, you were to revisit each of 
your CGI Scripts to change them to support your new model. At the 
moment, all of your code assumes that a list of athlete names or an 
AthleteList is returned from your model. You were to grab your 
pencil and amend each CGI to work with athlete IDs where necessary. 



Y^c>Ug e r »eedI-the "glob” ^dule, 

J\ 9 e C^ e pjVo*_sW0'' does 
all this work tov* you. 





c*t jr\arnes|D_A-om__s*fcoreO 

■i 1 dc \ 



print(yate.start_response ()) 

print(yate.include_header("NUAC's List of Athletes")) 
print(yate.start_form("generate_timing_data.py")) 

print(yate.para("Select an athlete from the list to work with:")) 
for each athlete insorted(athletes): 


The "athletes” are ww a list ot 
lists, so a»»e»d the tode to ^et 
at the data you «eed- 



l 


print (yat^ ^adio_button ^'which_athlete" , r ' l i a1wr;) < P 

print(yate.end_form("Select 

print(yate.include footer({"Home":/ "/index.html"})) 


eadh athleteCOJ, eadh a*thle*teCIJ) 


This is tt generate_t»mh^data.f/ 

#! /usr/local/bin/python3 


radio butbon idO ?J? 




import egi 

import athletemodel 

import yate 


t l-t looks like you »>ight «eed 
a sli<\htly ditfev-ent 'Vadio_ 

buttonO” -Irunttion?!? 


I h , e l Ct \ this 

Solutio» IS o» the »ext page 



4 eh the athletes data 
(rom the wodel, uihith 
returns a dittio»av-y- 



form_data = egi.FieldStorage() 

athlete_name = form_data['which_athlete'].value 

athlete — athletemodelget_athlete_^rom_id^athlete — id) 

print(yate.start_response()) 

print(yate.include_header("NUAC's Timing Data")) — athleter/Vaime ; J + * 

print (yate . header ( "Athlete : " + nTTfT r7 ’l ~ ~p i i i jn r^ T" 1 ' y DHP ~ JT i ntlgty L.~ ~ [ ~i tJVL~ 
print(yate.para("The top times for this athlete are:")) 
print (yate . u_list () ) athleteC^top^ 

print (yate . para ( "The entire set of tiir^n^TJSCa^is : " + str ( a^hfcl-ert^sTaW^ + 



td^ ' nrfwr " ed data as 

the diet a “ eMih 9 
9 at t he athlete s data. . 

: " + athleteC'DOB'3 


+ " • " ) ) 


tr(athleterdata ; 3) 



" (duplicates removed).")) 

print(yate.include footer({"Home": "/index.html", "Select another athlete": "generate list.py"})) 


you are here ► 


339 


















cgis for android 


^Jharpen your pencil 

Solutiori 

TWis is "tVic ^ 

"ae«evaie_na«>es.fY” 

C^l- 

#! /usr/local/bin/python3 

import json 

import athletemodel 
import yate 


Part 2: You wererYt done with that pencil just yet! In addition to 
amending the code to the CGIs that support your web browser's 
Ul, you also needed to change the CGIs that provide your webapp 
data to your Android app. You were to amend these CGIs, too. 


names 


^— > 9 et_*ames|D_Pror»_sWO 

athletemodel. 


print (yate . start_response ( 'applicatiori/j son' ) ) 
print(json.dumps(sorted(names))) 

/W beve is 
V»evaie_daia.p/' 

C(\ I- 





#! /usr/local/bin/python3 

import egi 
import json 
import sys 

import athletemodel 
import yate 


f ha *9« «eed io be r»ade 
to these C$ls, beeause your Ahdroid app 

VOT alP^Tu*/ ' h y ° M I WebappS da ^' 

W ' 3,1 ^bat genev-ated HTML. 



form_data = egi.FieldStorage() 

athlete_name = form_data['which_athlete'].value 

athlete — ath I ete r* od e I • 9 et_ath I ete^-Pr or*__i d (ath I ete__i d) 

print(yate.start_response('application/j son' )) 
print(json.dumps(athletes[athlete namej.as dict)) 


Add -tWis eode -to 
w yatep/ “to support 
tbe creatio* o-f v-adio 
butW that provide 
a value -for the 
butto* that diPCers 
-Pro** the butto* 

tc*t- 




A third argumcht lets you spc^i-Py 


* ID io 9 o with the ridio Ctti 


def radio__button_id (rb__name, rb__value 

return('Cinput type= M radio" name ^ 1 + rb_name + 

'" value="' +Cstr(rb id) + ' 5, > ' + rb value + '<br />') 
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Tqst Drwq 


Before you run your amended webapp, be sure to move you SQLite database into the top-level 
directory of your webapp (that is, into the same folder your index. html file). That way, your 
model code can find it, so move it into your webapp's root folder now. When you are ready, take your 
S-tav-t (ov SQL-powered webapp for a spin. 

res-tav-t) y ouV " 
web scvvcv-. 

i_ 




« rs n 


| File Edit Window Help StartYourWebEngine 

i 

$ python3 simple httpd.py 

Starting simple httpd on port: 8080 



Welcome to the National Underage Athetics Committee's Website 



< 

+ | (khttp://localhost: 8080 /index.html 

c 

tOr Google 

1 

□□ 

:::: Apple Yahoo! Coogle Maps YouTube Wikipedia Popular^ 








The NUACs List of Athletes 



◄ 


+ 

(khttp://localhost:8080/cgi-bin/generate_list.py 

C (Oj Google 


0P 

■ ■■■ 
■■■■ 
■■■■ 

Apple Yahoo! Google Maps YouTube Wikipedia Populare 



The NUACs List of Athletes 


Welcome to the NUACs Website. 

Here is our athlete’s timing data. Enjoy! 

See you on the track! 


Select an athlete from the list to work with: 

O James Lee 
O Julie Jones 
O Mikey McManus 
© Sally Sanchez 
O Sarah Sweeney 
O Vera Vi 


Select 



Clitk on -the 
linlc on -the home 
? ay. 


And thev-e s 
Sally S tintina 
data. 



#4 O ^ NUACsViming Data 

◄ ► | + (khttp://localhost:8080/cgi-bin/generat^ 

timing_data.py C (Q^ Google 

DQ :::: Apple Yahoo! Google Maps YouTube Wiki 

edia Popular^ 

NUACs Timing Data ) 


Athlete: Sally Sanchez, DOB: 2002 -V 1 - 24 . 


The top times for this athlete are: 

• 2.11 
• 2.26 
• 2.31 

The entire set of timing data is: ['2.11', '2.26', '2.31’, '2.32', '2.41*, '2.44', ’2.51‘, '2.55', '3.00', '3.01'] 
(duplicates removed). 

Home Select another athlete 






Display the list 
o-f athlete names 
v~adio btrkfcohs. 


That worked well. But what about your Android app? 
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amend for android 


You need to amend your Android app, too 

Unlike your HTML-based webapp, where all of your code resides and is 
executed on your web server, your Android app runs on your phone and it is 
programmed to work with a list of names, not a list of names and athlete IDs. 

When you run coachapp . py on your emulator, weirdness ensues... 




TWis is y/eiv-dAmstead o£ 
i\\t names, youv aw> * 

disylayi^ a list ot list* 





Ahd it you fa r «Selost- you, app 

wi-th a Valu e £W". Bul e ,. 




5554:droid2.2 


®HD<^ 12:30 


Here is your list of athletes: 

["James Lee”,1] 

(•) 

["Julie Jones",4] 

O 

["Mikey McManus",6] 


["Sally Sanchez",5] 

( ( | 

["Sarah Sweeney",2] 


Select 

- 

Quit 

_ 

_ 


Just like with the GGI Scripts, you need to amend you Android app to work 
with the data that’s now arriving from your web server—that is, a list of lists 
as opposed to a list. 

That shouldn’t take too long, should it? 
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Here is your current coachapp. py code, which you need to amend to support the way your 
webapp’s model now works. Grab a pencil and make the necessary changes to this code. 


import android, json, time 


from urllib import urlencode 
from urllib2 import urlopen 


hello_msg 
list_title 
quit msg 
web server 
get_names_cgi 
get_data_cgi 


"Welcome to NUAC's Timing App" 
'Here is your list of athletes:' 
"Quitting NUAC's App." 

'http://192.168.1.34:8080' 

'cgi-bin/generate_names.py' 

'/cgi-bin/generate_data.py' 


def send_to_server(uri, post_data=None): 

# There is no change to this code from the previous chapter. 


app = android.Android() 


def status_update(msg, how_long=2): 

# There is no change to this code from the previous chapter. 


status_update(hello_msg) 

athlete_names = sorted(json.loads(send_to_server(web_server + get_names_cgi))) 

app.dialogCreateAlert(list_title) 

app.dialogSetSingleChoiceltems(athlete_names) 

app.dialogSetPositiveButtonText ( 'Select') 

app.dialogSetNegativeButtonText('Quit') 

app.dialogShow() 

resp = app.dialogGetResponse().resuit 

if resp['which'] in ('positive'): 

selected_athlete = app.dialogGetSelectedltems().resuit[0] 
which_athlete = athlete_names[selected_athlete] 

athlete = json.loads(send_to_server(web_server + get_data_cgi,{'which_athlete': which_athlete})) 

athlete_title = athlete['Name'] + ' (' + athlete['DOB'] + '), top 3 times:' 

app.dialogCreateAlert(athlete_title) 

app.dialogSetltems(athlete['Top3']) 

app.dialogSetPositiveButtonText('OK') 

app.dialogShow() 

resp = app.dialogGetResponse().resuit 
status_update(quit_msg) 
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support the new model 



ExwctSe 

SoUitiOH 


Here is your current coachapp. py code, which you need to amend to support the way your 
webapp’s model now works. You were to grab a pencil and make the necessary changes to this 
code. 


import android, json, time 
from urllib import urlencode 
from urllib2 import urlopen 


hello_msg 
list_title 
quit msg 
web server 
get_names_cgi 
get_data_cgi 


"Welcome to NUAC's Timing App" 
'Here is your list of athletes:' 
"Quitting NUAC's App." 

'http://192.168.1.34:8080' 

'cgi-bin/generate_names.py' 

'/egi-bin/generate_data.py' 


def send_to_server(uri, post_data=None): 

# There is no change to this code from the previous chapter. 


app = android.Android() 


def status_update(msg, how_long=2): 

# There is no change to this code from the previous chapter. 


£*bcati ihe athlelc 
»ar»es ONLY 4or» ih e 

lisi o-p lis-fcs. 


alblcies — 



status_update cTjnsg) 

sorted (j son . loads (send_to_server (web_server + get_names_cgi) ) ) 
app . dialogCreateAlert (list_ti^^e*) "*—s athlete^ames =■ CathCOJ -for a*th 'm a*thlc*tcs3 

app.dialogSetSingleChoiceltems(athlete_names) 

app . dialogSetPositiveButtonText ( ' Select' ) v». TVlS is £ tool USC O-p £ 

app.dialogSetNegativeButtonText (' Quit ') ^ompvchchsioh. 

app.dialogShow() 

resp = app.dialogGetResponse().resuit 

PcWme IP ass©6iated 
-the sclcticd athlete- 


if resp['which'] in ('positive'): 




v/r 


selected_athlete = app.dialogGetSelectedltems().resuit[0] 

which_athlete - athletesUeledted__athlete JCI 3 

athlete = json.loads(send_to_server(web_server + get_data_cgi,{'which_athlete': which_athlete})) 
athlete_title = athlete['Name'] + ' (' + athlete['DOB'] + '), top 3 times:' 

app. dialogCreateAlert (athlete^rtle ) 

app. dialogSetltems () athletertop3 ; 3 
app.dialogSetPositiveButtonText('OK') 
app.dialogShow() 

resp = app.dialogGetResponse().resuit 
status update(quit msg) 



h s">all adjusWh-fc io neY .i 


heeded Access ihe 

a-t-tv-ibu-te- 
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Poo] Puzzjc 

Your job is to take the code from the pool and place it into 
the blank lines in the program. Your goal is to write 
the code to have your app provide the user with a 
mechanism to add a timing value to the server for 
the currently selected athlete. For now, send your 
data to the cgi-bin/add_timing data . py 
CGI script. 

Hint: the code from get2inputsapp.py (from earlier in 
this chapter) should come in handy here. 

/\dd anothev -to h>e 

\{, vev-sio* <A 70UV 3 ^p- 


app.dialogSetNegativeButtonText('Add Time') 


dialo^ m ite twrvw 


if resp['which'] in ('positive') 


pass 



,av,er do *>otW$ 1 ? 
e,t . . -n. 4-kr wsev- 




send to server(web server + add time egi,{'Time': new time, 'Athlete': which athlete}) 


/-P some ihpu-fc is 
su Ppli ed> schd i-fc 
web sev-vev” 
kge-fcheir with the 
athlet es ID. 
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allow android input 



^ncboicl P^a] puzzjc §o]u£ien 

Your job was to take the code from the pool and place it 
into the blank lines in the program. Your goal was 
to write the code to have your app provide the user 
with a mechanism to add a timing value to the server 
for the currently selected athlete. For now, you 
were to send your data to the cgi-bin/add_ 
timing data . py CGI script. 


Hint: the code from get2inputsapp.py (from earlier in 
this chapter) should come in handy here. 


app.dialogSetNegativeButtonText('Add Time') 


if resp['which'] in ('positive’): 
pass 

elif resp['which’] in ('negative'): 

timing title = 'Enter a new time' 

timing msg = 'Provide a new timing value ' + athlete['Name'] + 
add time egi ='/cgi-bin/add_timing_data.py' 



De-Pine -the dialogs 
tities and speci-fy 
the C$l to send 
the data to. 


resp = app.dialogGetlnput(timing title, timing msg).resuit . i 

if resp is not None: — SOwC vSCV" W ?' A 

new time = resp 

send to server(web server + add time egi,{'Time': new time, 'Athlete': which athlete}) 
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Use the tools/adb command to copy your latest app to the emulator, and give your app a go. 


fk r> 


SSS4 droid2.? 



© © 



SSS4 drt>id2.2 


Seledt u Vcra w 
-(Vorn tbe list 

(J( 


—to S CC 1 /eirVs top 3 

tn^CS; then tap -the 
w Add Ti»e" butW.. 


Great: your data 
is sent from your 
Android app to 
your web server. 


...-to entev- a new 
tir*e> v/WidVi 'is 
tben sent to 
youv- web servev- 


V 


I Fila Edit Window Haln DataFrnmAndrnid 


$ python3 simple_httpd.py ] 

Starting simple_httpd on port: 8080 I 

198.162.1.34 - - [27/Sep/2010 14:51:47] "GET /cgi-bin/generate_names.py H^/p/1.1" 200 - 

198.162.1.34 - - [27/Sep/2010 14:52:01] "POST /cgi-bin/generate_data. b^f^TTP/1.1" 200 - 

198.162.1.34 - - [27/Sep/2010 14:52:19] "POST /cgi-bin/add_timing_d^L<py HTTP/1.1" 200 

localhost, 198.162.1.34, Mon Sep 27 14:52:19 2010: POST: Athlete->3 Time->1.33 


9 
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database update 


Update your SQLite-based athlete data 

All that 5 s left is to change the cgi-bin/add_timing data . py CGI 
script to write your submitted data to your database, as opposed to the web 
server 5 s console screen. 

At this point, it 5 s a trivial exercise, because a single SQL INSERT statement 
will do the heavy lifting. 


«OO 


add timing data.py - /Users/barrvp/HeadFir5tPython/chapter9/cgi-bin/add timing data.py 


#1 /usr/local/bin/python3 

import egi 
import sqlite! 

import yate 

print (yate.start_response( 1 text/plain' )) 

form = egi.FieldStoragef) 
the_id = form data[ 1 Athlete 1 ].value 
the time = form data[ 'Time' ].value 


the data seht to 

|jy*/Mnr.u r. A 


ex Xhc data scht to youv wcb 
rowseif Wor» your A*droid app. 


connection = sqlite3.connect( 1 coaehdata.sqlite ' ) 
cursor = connection.cursor() 

cursor . exeeute( "INSERT INTO timing_data (athiete_id f value} VALUES (? f 7) H f 

(the _id, the_ time) ) 

connection. commit( ) 

connection.closeM *\ , h , i ^ 

1 ‘ t_INSERT the data mto your 

print ( 'ok. ' ) "tWmg^data” talole- 


Ln: 22 


Coh 0 


With this version of your CGI script running on your web server, any new 
times entered by anyone on an Android phone are added to the data in the 
database. 

The NUAG no longer has to worry about adding data to text files, because 
the files are effectively obsoleted by the use of SQLite. 

YouVe produced a robust solution that is more manageable, scalable, 
programmable, and extendable. And it 5 s all thanks to the power of Python, 
it 5 s database API and the inclusion of sglite3 in the Standard library. 


All that’s left to do is sit back, relax and bask in the 
glory of your latest programming creation... 
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manage your data 


The NUAC is over the moow! 


Of course, your use of SQLite gives you more than just easy insertions 
of data. With the NUACfs data in tables, it 5 s easy to answer some of the 
questions that have been on their mind. 



To answer these and other queries on the data in the NUAC 5 s database, you’11 
have to bone up on your SQL. Then it 5 s up to you to take it from there. 

You Ve converted your webapp to use an SQL database. As your data 
management needs increase, you can consider alternative heavy-duty data 
management technologies as needed. 

This is great work. Your webapp is ready for the big time. 
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python toolbox 



Your Pythow Toolbox 

You’ve got Chapter 9 under your 
belt and you’ve added some 
key Python tools to your evey 
expanding Python toolbox. 


wtto 




St 


ohe o\r 


s ta^a'rd'^ d Q "^ asc i data' 03 

a Tl- ^ 3 

swste»* * ro 

yvo^a*»" 


Database Lingo 

# "Catabase" - a tollediio» o-P 
»»oire iables. 

• "Tabi/ - a tolle,*» of 

*r»#i a, « «. „«« 

£olur*hS. 

9 - Oie "SWWtd <$uery 

Lanjuagc" is ihc Uguage o-f the 
database woirld and ii Icis you v/ov-k 

7J *"* d3 ^ i» your daiabase usi»g 
staiemehis sudh as CREATE, f/VSERT 

a^d SELECT. 





BULLET POINTS 


The fieldstorage () method from 
the Standard library’s egi module lets 
you access data sent to your web server 
from within your CGI script. 

The Standard os library includes 
the environ dictionary providing 
convenient access to your progranis 
environment settings. 

The SQLite database system is included 
within Python as the sglite3 Standard 
library. 

The connect () method establishes a 
connection to your database file. 

The cursor () method lets you 
communicate with your database via an 
existing connection. 

The exeeute () method lets you send 
an SQL query to your database via an 
existing cursor. 

The commit () method makes changes 
to your database permanent. 

The rollback () method cancels 
any pending changes to your data. 

The close () methodclosesan 
existing connection to your database. 

The “?” placeholder lets you parameterize 
SQL statements within your Python code. 
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10 sccj!ng your Webapp 





It ali started with the internal 
combustion engine, then it was 
the electric engine, and now 
theres App Engine. Will this 
torture never end? * 


The Web is a great place to host your app...until things get real. 

Sooner or later, you’ll hit the jackpot and your webapp will be wildly successful. When that 
happens, your webapp goes from a handful of hits a day to thousands, possibly ten of 
thousands, or even more. Will you be ready? Will your web server handle the load? How 
will you know? What will it cost? Who will pay? Can your data model scale to millions 
upon millions of data items without slowing to a crawl ? Getting a webapp up and running 
is easy with Python and now, thanks to Google App Engine, scaling a Python webapp is 
achievable, too. So...flip the page and find out how. 


this is a new chapter 
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a whale of data 


There are whale sightings everywhere 

The Head First Whale Watching Group (HFWWG) coordinates the live 
cetacean sightings for the entire country. To date, theyVe provided a PDF 
form on their website that members of the public can download, fili in, and 
mail to the HFWWG Central office. 

The form contains the essential data needed to record the sighting: 


Head First Whale Watching Grouo 
Casual Sighting Form 9 P 


HFWWG.ORG 



Fin Type: 


Blow Type: Tali 


Falcate 

Tri angular 

Rounded 

Humpback 

Orca 

Blue 

Beluga 

Fin 

Gray 

Tali 

Bushy 

Dense 

Flat 

Small 

Moderate 

Large 

Breaking 

High 


Killer 


Spem) 


After a busy sightings weekend, the Central office is swamped with completed 
forms for thousands of sightings.. .which is a data-entry nightmare as all those 
forms can take an age to process manually. There’s nothing worse than being 
stuck in front of your computer entering data when all you want to do is be 
out on the water looking for humpbacks... 
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scaling your webapp 


The HFWW(r weeds to automate 




Ideally, a solution that works on 
the Web would be great. That 
way, anyone f rom anywhere 
could record a sigh+ing. Look! 
Theres one... 


O 



Suggesting to the HFWWG that they invest in an expensive web hosting 
solution isn’t going to make you any friends. It 5 s way too expensive to buy 
the capacity they’11 need for the busy weekends and a total waste of capacity 
when sightings are infrequent. 

Suggesting that they invest in a large, state-of-the-art web server that can be 
hosted in the Central office is also a nonstarter: there’s no one to look after a 
setup like that, and the broadband link required to handle the anticipated 
traffic would blow the their budget right out of the water. 


Is there another option? 
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enter app engine 


Puild your webapp with G-oogle App Engine 


Google App Engine (GAE) is a set of technologies that lets you host your 


webapp on Google’s cloud computing infrastructure. 

GAE constantly monitors your running webapp and, based on your webapp’s 
current activity, adjusts the resources needed to serve up your webapp 5 s pages. 
When things are busy, GAE increases the resources available to your webapp, 
and when things are quiet, GAE reduces the resources until such time as 
extra acti vi ty warrants increasing them again. 

On top of this, GAE provides access to Google’s BigTable technology: a set of 
database technologies that make storing your webapp’s data a breeze. Google 
also backs up your webapp’s data on a regular basis, replicates your webapp 
over multiple, geographically dispersed web servers, and keeps App Engine 
running smoothly 24/7. 

And the bestport? GAE can be programmed with Python. 

And the even better part? You can start running your webapp on GAE for free. 



Five million page views? That 5 s a lot of sightings... 


charge and will continue to do so until your webapp 
processes five million page views per month. Once it 
exceeds this threshold, you’11 need to pay Google for 
the extra capacity used. If you never reach the limit, 
your use of GAE is not charged. 


Initially, there isn’t one. 

Google provides this webapp hosting Service at no 
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Pownload and install App Engine 

When your webapp is ready for deployment, you’11 upload it to the Google 
cloud and run it from there. However, during development, you can run a test 
version of your webapp locally on your computer. All you need is a copy of 
the GAE SDK, which is available from here: 



http://code.google.com/appengine/ A-Piev ihs-fcalla-k' W j 

^ddcd io -thciv 

Download the GAE Python SKD for your operating system. Windows, Mac / uc 

OS X, and Linux are all supported, and installation is straightforward. 

On Uw»%, a ne* toldev talled 

^ ,s areated a 

suttess-ful mslall. 

(rAE uscs Python Z.5 

The version of Python built into GAE is a modifled version of the Python 2.5 
release. As when you worked with Python for Android, the fact that you aren’t 
running Python 3 isn’t such a big deal with GAE, although you do need to 
ensure Python 2.5 is installed on your computer. Open up a terminal window 
and type: 


python2.5 -V 


If this command gives an error, pop on over to the Python website and grab 
the 2.5 release for your operating system. 


thereicffe no 

Diimb Quespons 


Arerft things going backward here? First, there was Python 3, then it was Python 2.6 for Android, and now we are dropping 
down to 2.5 for App Engine? What gives? 

Thafs a great question. Ifs important to remember to always code to the restrictions placed on you.You might think that it sucks that GAE 
runs on Python 2.5, but you shouldnl Think of it as just another restriction placed on the code you write—that is, it must target Release 2.5 
of Python. As with the Android code you created in the previous chapters, the GAE code you are about to write is not all that different than the 
Python code for 3. In fact, you will be hard pressed to spot the difference. 
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testing app engine 


Make sure App Engine is working 

The environment supported by GAE within the Google cloud supports 
Standard GGI or Python 5 s WSGI. To build a GAE-compatible webapp, you 
need three things: a folder to hold your webapp’s files, some code to execute, 

and a configuration file. 

To test your setup, create a folder called mygaetest. Within the folder, 
create a small GGI you can use to test GAE. Call this GGI sayhello . py. 
Use this code: 



The configuration file must be called app . yaml and it, too, must be in 
your webapp 5 s folder. This file telis the Google cloud a little bit about your 
webapp’s runtime environment. Here 5 s a basic configuration file: 


Ii doesn't get »»udh 
easieir than -this...a 
plain-text message i s 
displayed within your 
browser whcnevcv 
■this C6/I runs. 



TV>e "applitation” ' me 
your webapp and is the san>e na*»e 

as your tolder- 

"runti»e" -telis that your webapp J 

is in and will \run on Python 

Tbink ot the “handlers" sedtion 

the doh+.guiration tile as a top-| eve | 

webapp routing medhanis». 


applicationi mygaetest 
version: 1 
runtime: python 
api_version: 1 

handlers: 

- uri( /.* 

script: sayhello.py 



current vers.on o+ y OU r webapp 
(and usually starts at I). 

The "api__yetsion" 

inditates the telease a 
you are targeting- 




This entv-y telis £jA£ to v-oute 
all reguests to youv- webapp to 
your "sayhello.py" progra*». 



Go ahead and create the folder called 
mygaetest and the two files shown here. 



sayhelloTY, 


app.yarj 
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'esT DriVq 


Cli£k “this 

butfcoh io 
siari youv- 
webapp. 


The GAE SDK includes a test web server, so let's use it to take your test GAE webapp for a spin. If you 
are running on Windows or Mac OS X, fire up the Google App Engine Launcher front end.This tool 
makes it easy to start, stop, and monitor your webapp. On Linux, you'll need to invoke a command 
to kick things off. If you are using the GAE Launcher, choose File -> Add Existing Application from 
the menu system to browse and select your webapp's folder. Also: be sure to edit the Launcher's 
Preferences to select Python 2.5 as your preferred Python Path. 



CoogleAppEngineLauncher 


Stop 


Logs SDK Console 


£ 

Edit 



Deploy Dashboard 


Name 


Path 


/Users/barryp/HeadFirstPython/chapterlO/mygaetest 


Port 


8080 



TW is ho giraphidal 4Vo*»t 

C)r\d toit Linux, so s^fcav-fc 

y°uv webapp 4Vor» -the 
tomma^d line. 




I File Edit Window Help GAEonLinux 


$ python2.5 google_appengine/dev_appserver.py mygaetest/ 

INFO 2010-10-02 12:41:16,547 appengine_rpc.py:149] Server: appengine.google.com 

INFO 2010-10-02 12:41:16,555 appcfg.py:393] Checking for updates to the SDK. 

INFO 2010-10-02 12:41:17,006 appcfg.py:407] The SDK is up to date. 

WARNING 2010-10-02 12:41:17,007 datastore_file_stub.py:657] Could not read datastore data 
from /tmp/dev_appserver.datastore 

INFO 2010-10-02 12:41:17,104 dev_appserver_main.py:431] Running application mygaetest 
on port 8080: http://localhost:8080 


L 


Tbis is bow ‘tbe 
Laundbev 

looks oy \ Mad 
)(...i-t looks sirrxilav* 
oy\ Mndows. 

With your webapp running and waiting on port 8080, open your favorite web browser and surf on 
over to the http : //localhost: 8080/ web address. 


/\nd “tbcYC vt lS -- 
■the w>essa$e ww» 
vowv -test vieba^! 


« n n 


http://localhost: 8080 / 


◄ 

* 

+ 

http://localhost:8080/ C 


C\ (Q* Google 


0 




PQ 


Apple Yahoo! Google Maps YouTube Wikipedla Popularv 


Helio from Head First Python on GAE! 
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more work? 



I don'+ believe it. This is 
actually more work than 
plain old CGI...and youre 
claiming this is better?!? 


Yes, it is more work. But that’s about to change. 

For now, this is more work than you’re used to, but remember that 
this is just a quick test to make sure your GAE test environment is 
up and running (and it is). When you start to work with some of 
GAE’s web development features, you’11 initially see that there 5 s a 
lot more going on behind the scenes than meets the eye. 
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App Engine uses tbe MVC pattern 


Google has built GAE to conform to the familiar Model-View-Gontroller 
(MVG) pattern. 

Like your webapp from the previous chapter, the model component of a 
GAE-enabled webapp uses a back-end data storage facility that’s known as 
the datastore. This is based on Google 5 s BigTable technology, which provides 
a “NoSQL’ ? API to your data, as well as a SQL-like API using Google 5 s 
Query Language (GQL). 

GAE’s views use templates, but unlike the simple string templates from the 
previous chapter, GAE uses the templating system from the Django 
Project, which is one of Python’s leading web framework technologies. In 
addition to templates, GAE includes Django 5 s forms-building technology 

And, of course, any controller code is written in Python and can use the 
GGI or WSGI standards. Unfortunately, you can’t use your yate module 
with GAE, because it is a Python 3 library (and would need to be extensively 
re written to support Python 2). Not to worry: the facilities provided by GAE 
“out of the box” are more than enough to build great webapps. 



Tbe Oa.'r°" e ' r 


Tbe Model 


The |/i 


e w 



So. Jike any other webapp that I build, 
with App Engine I def ine a model for my data, 
create some templates for my view, and then 
control it all with code, right? 


Yes, it’s the same process as any other webapp. 

Google has worked hard to ensure that the move to App Engine 
is as painless as possible. If you understand MVG (as you now 
do), you are well on your way to creating with GAE. It’s just a 
matter of working out how GAE implements each of the MVG 
components. 
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model data 


Model your data with App Engine 


App Engine refers to data items stored within its datastore as properties , which 
are defined within your model code. 


Think of properties as a way to define the name and types of data within 
your database schema: each property is like the column type associated piece 
of data stored in a row, which App Engine refers to as an entity. 




As with traditional SQL-based databases, your GAE datastore properties are 
of a specific, predeclared type. There are lots to choose from, for instance: 

• db.StringProperty: a string of up to 500 characters 

• db.Blob: a byte string (binary data) 

• db.DateProperty: a date 

• db.TimeProperty: a time, 



fov tbe tuli lisfc 

tyyes suyyovted, 0|r > 

to Utp //todey>o$leW 

a f? er,oj'me/does/y7-tV>or,/daUsW/ 

iTyesa^dyvoyevivtlasses.bt™l a«d 
take a lo°k- 


db.IntegerProperty: a 64-bit integer 


db.UserProperty: a Google account 


■fhe yv-evoiws tV»apW' r 


TWis data is stov-ed as ^ 
a K db.|y>tey*-P<royeaY • 


SWe this da-ta as 
"dbStvingPvopevty”- 


James Lee 
Sarah Sweeney 
Vera Vi 
Julie Jones 
Sally Sanchez 
Mikey McManus 


2002-03-14 
-06-17 
2002-12-25 
2002-08-17 
2002-11 -24 
2002-02-24 


This data is stov-ed as 
a db PatePv-opfv-ty”. 
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|mport *the W db 
module -fv-ow» “the 

ey.-teinsiovis. 


Eaeh pv-opev-ty 
is assighcd io 3 


Kdme. 


P 00 ] ptizzjc 


Your job is to take the properties from the 
pool and place them in the correct 
place in the class code, which is in a 
file called hfwwgDB . py. Your goal 
is to assign the correct property type 
to each of the attributes within your 
Sighting class. 




from google.appengine.ext import db 


class Sighting(db.Model) 
^ name = 
email = 
date = 
time = 
location = 
fin type = 
whale_type = 
blow_type = 
wave type = 
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property types 



P00] puzz]c 


Your job was to take the properties from 
the pool and place them in the correct 
place in the class code, which is in a 
file called hfwwgDB . py. Your goal 
was to assign the correct property 
type to each of the attributes within 
your Sighting class. 


from google.appengine.ext import db 



class Sighting 
name = 
email = 
date = 
time = 
location = 
fin type = 
whale_type 
blow_type 
wave type 


(db.Model): 

db.StringProperty() 

db.StringProperty( 
db.DateProperty() 
db.TimeProperty() 
db.StringProperty() 
db.StringProperty() 
= db.StringProperty() 
= db.StringProperty() 
= db.StringProperty() 



Evevy-thmg is a 
"S-t\rihgPirope»"ty", 
excepi the "date" a*d 
time" -fields. 
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scaling your webapp 


What good is a wodel without a view? 

GAE not only lets you define the schema for your data, but it also creates 
the entities in the datastore. The first time you go to put your data in the 
datastore, GAE springs to life and makes room for your data. There 5 s no 
extra work required by you, other than defining your model in code. It’s 
useful to think of GAE as executing something similar to a SQL CREATE 
command on the Jly and as needed. But how do you get data into the GAE 
datastore? 



The short answer is that you put it there , but you first need to get some data 
from your webapp’s user.. .and to do that, you need a view. And views are 
easy when you use templates. 

App Engine templates in an instant 

Recall that the templating technology built into GAE is based on technology 
from the Django Project. Django 5 s templating system is more sophisticated 
than the simple string-based templates used in the previous chapter. Like your 
templates, Django’s templates can substitute data into HTML, but they can 
also execute conditional and looping code. 

Here are four templates you’11 need for your HTWWG webapp. Two of them 
should be familiar to you: they are adaptions of those used in the previous 
chapter. The other two are new. Go ahead and grab them from this book’s 
support website. As you can see, rather that using the $name syntax for 
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use a template 


Use templates in App Engine 

To use a template, import the template module from google . 
appengine . ext. webapp and call the template . render () function. 
It is useful to assign the output from template . render () to a variable, 
which is called html in this code snippet: 


from google. appengine. ext. webapp import template As^usudl, s-fcd\rt wrth youv 

html = template.render('templates/header.html ', {'title': 'Report a Possible Sighting 



Call u -ter»pla-te. 
rehdeirO"... 


..supplying the 
template na^e. 





...as viell as a ditWavY tbat ">a?s vatas b> 
Te «a«>ed iem^late variable- 


This is similar to the mechanism your yate . py module uses to parameterize 
the data displayed within your HTML pages. 



Yes, create your view with templates. 

Just like the other webapps that youVe built, you 
can create your view in much the same way using 
Python code. It’s a bummer that you can’t use your 
yate . py module, but Django 5 s templates provide 
most of the functionality you need here. 


tfieretqre no 

Dumb Questipns 



Should I create one big template for my entire web page? 


You could, if you want. However, if you build up your view from snippets of HTML in templates, you open up the possibility of reusing those 
HTML snippets in lots of places. For instance, to maintain a consistent look and feel, you can use the same header and footer template on all of 
your web pages, assuming of course that your header and footer aren’t already embedded in an entire web page (which can’t be reused). 
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scaling your webapp 



4 k Let's write the rest of the code needed to create a view that 
displays a data entry form for your HFWWG webapp. 


TWis todc 5 oes a 
neu w-oy-a» talled 


In addition to your web page header code (which already exists 
and is provided for you), you need to write code that starts a new 
form, displays the form fields, terminates the form with a submit 
button, and then finishes off the web page. Make use of the 
templates youVe been given and (here's the rub) do it all in no 
more than four additionol lines of code. 



from google.appengine.ext.webapp import template 


html 


html 


template.render('templates/header.html', {'title': 'Report a Possible Sighting'}) 

fwtttdI tirc £«to,ts of ■hfc.r 

„«.! . of HTML 


Rc^ci^bcv: Y\0 

move 

Imcs o£ 6odc| 




Now that you have attempted to write the code required in no more than four lines ofcode, what 
problem(s) have you encountered. In the space below, note down any issue(s) you are having. 
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data-entry display 



i | You were to write the rest of the code needed to create a view 
that displays a data entry form for your HFWWG webapp. 

In addition to your webpage header code (which already exists 
and is provided for you), you were to write code with starts a 
new form, displays the form fields, terminates the form which a 
submit button, then finishes off the webpage. You were to make 
use of the templates youVe been given and (here's the rub) you 
had to do it all in no more than four more lines ofcode. 


from google.appengine.ext.webapp import template 

html = template.render('templates/header.html', {'title': 'Report a Possible Sighting'}) 

The VndorO" Wfcon alwa ys expetis L,». i if 

. 

templaterendev-^templates/'fo\rm^_S*ta\rt*h tm I*, {}) 


html = html + 



TMs is an issue, 
isnt it? 



html =■ html + templatev-endevt'templates/toV-m_endhtmr, { 1 sub_titl t' 'Submit Sighting}) 
html =■ html + template* V-ender^templates/tooterhtmr, rimks': "}) 



Having attempted to write the code required in no more thon four lines ofcode , you were to make a 
note of any issue(s) you encountered. 


This is J/yjPOSSlBLE to do in.just tour Jmcs ot Codt, bedcjuse th.eyV.s no way 
tp .generate .the .P0R/V1. tields that.I.need*. J; ean*t.eyen use.th.e. u dq^ J:?r 


.*f wftiptt.£y\om . u yate*py ; j. b.edause. that .epde. is npt. £pmpatible. vyith. Python Z.?.... 

—• — lik ' «*-■«-•»» rf tora, to M 

't/ “n of «■“ k~k «t Ih. i. fraLfel® 
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scaling your webapp 


Wouldn't it be dreamy if I could avoid 
hand-coding a <FORM> and generate the 
HTML markup I need f rom an existing data 
model? But I know it's just a fantasy... 




you are here ► 


367 






more borrowing from django 


Pjango's form validation framework 

Templates aren’t the only things that App Engine “borrows” from Django. 

It also uses its form-generating technology known as the Form Validation 
Framework. Given a data model, GAE can use the framework to generate the 
HTML needed to display the form 5 s fields within a HTML table. Here’s an 
example GAE model that records a person’s essential birth details: 


TWis tode is i* a tile 
talled tt WthDBr/'- 


— 

from google.appengine.ext import db 

class BirthDetails(db.Model): 

name = db.StringProperty() 

date_of_birth = db.DateProperty() 
time_of_birth = db.TimeProperty() 


This model is used with Djangcfs framework to generate the HTML markup 
needed to render the data-entry form. All you need to do is inherit from a 
GAE-included class called djangoforms .ModelForm: 


from google.appengine.ext.webapp import template 
from google.appengine.ext.db import djangoforms 
import birthDB 



| m?0 »t W libvavy m additio» 
{o y 0 u» 6\f\t data model- 


Create a »ew elass by i»he»iti» 9 t»om the 

elass, a»d the» li»k v< 
dlass -to you» data model- 


class BirthDetailsForm(djangoforms.ModelForm) 
class Meta: 

model = birthDB.BirthDetails 


you\r hcw 



— template.render('templates/header.html ', { 'title': 'Provide your birth details'}) 

html + template.render ('templates/f orm_start.html' , {}) ^ fY/ ^| ass ^cvatc 

1 « T | f | / ■ J_ <1_ l ■ rn — 0 . ■ m __ — » » I A 


= html +1 str (BirthDetailsForm (auto_id=False) ) 


youv- 


-Cov**- 


html + template. render ('templates/ form_end.html ' , {'sub_title': 'Submit Details'}) 
html + template.render('templates/footer.html', {'links': ''}) 


r l . U, 4- (W-t vjo»»V : Vow'11 9 et -to it i» 
-n, e « “ ~- <*«****']* k fceW **" 

a moment- Y<* noxp ^ validatio» t»amev-o»k. 

m odel the v.cw tode, a»d the 
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Check your forw 


The framework generates the HTML you need and produces the following 
output within your browser. 



“the 

yvcbtiest web 
CVCV" made, bu*t 
\y/OV"ks. 



Use the View Source menu option within your web browser to inspect the 
HTML markup generated. 


* rs O 


Source of: http://localhost:80Sl/ 


hames 


<hcad> 

<titlo>Provide your birth detailo</titlc> 

</hcad> 

<body> 

<hl>Provide your birth details</hl> 

<form method "POST" action=" /'> 

<tablo> 

<trX j th>Waaie : </th>ctdXinput type^" tcxt i; nante = 11 nane" /></td></tr> 

<tr><th>Date of birth :</th><td><input type j "text" naaie^date^^birth 11 /></fcd></fcr> 

■Time of birth :</th><td><input typej"tcxt !l namc ^ 11 t.ime_of_birth" /></tdx^tr><tr> 
tfbsp; </th><td><input type-*" submit" value®" Submit Details"></td></tr> 

</tabie> 

</£ornb- 

<px/p> 

</body> 


By ■*b_id-1, w in ^ ^ tht 
<•* ^ 


A 


tl, t), a «ao tv-amewovk is srnari h, £ yf ate j 

ItKU « 1 . j r* ^ * 

M» e «ames wsed 'm 


lt’s time to tie things all together with your controller code. 
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controller code 


Controlling your App Engine webapp 


Like your other webapps, it makes sense to arrange your webapp controller 
code within a specific folder structure. Here’s one suggestion: 




Put a!l o-P your webapps 
Controller todt and 
Coh-Piguration -Piles ih here. 



|£ you have statiC Content, 
pu-t it m here (at the momert, 
-tVns -folder is er»fty)- 





Put your HTML 
templates ih here. 


As you Ve seen, any GGI can run on GAE, but to get the most out of Google’s 
technology, you need to code to the WSGI Standard. Here’s some boilerplate 
code that every WSGI-compatible GAE webapp starts with: 


|w>port Ayy 
Evires 
u webapy elass. 


ITT 


Create ah 
hew u webapp w 


objeet 

-foir youv 
applidatio» 


rom google.appengine.ext import webapp 
from google.appengine.ext.webapp.util import run_wsgi app 



a u-tili-ty -that 
ruhs your webapp. 


class IndexPage'webapp.RequestHandler): 

TW.s «tttod «h» a web «V** 
is vedeWed V>y you»- webapp- 



def get(self) : 
pass 



app = webapp.WSGIApplication([('/•*',.IndexPage )], debug=True) 


def main(): 

run_wsgi_app(app) 


S-tavt youv webapp- 


if 


_ name == '_main _' : 7 s , 

main () J. J«t us e thes e iwo li»« ot dode as _ is . 



This dlass 
respohds to a 
web re<\uest 
^rom your web 
browser. 

7~his is 

ho t u»lik e 


s witch 


^4/ iratk 


»hg oh 
ihg. 
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scaling your webapp 



App Engine Code Magnets 

Let's put everything together. Your model code is already in your hfwwgDB . py 
file. AII you need to do is move that file into your webapp's top-level folder. Copy 
your templates folder in there, too.Your webapp's controller code, in a file called 
hf wwg. py, also needs to exist in your top-level folder. The only problem is that 
some of the code's all over the floor. Rearrange the magnets to fix things. 


from google.appengine.ext import webapp 

from google.appengine.ext.webapp.util import run wsgi app 

from google.appengine.ext import db 

from google.appengine.ext.webapp import template 

from google.appengine.ext.db import djangoforms 



AH o*£ itopoifks 
have swrvivecL.so 
■cheve s no need lo 
'rearrange Ihem. 


Lei's besi hovi well 
you've bee* 
allenbon* Theres no 
&uid‘m^ liwcs on l^e 

door 


= template.render('templates/header.html', {'title': 'Report a Possible Sighting'}) 

= html + template.render('templates/form_start.html', {}) 

= html + template.render('templates/form_end.html', {'sub_title': 'Submit Sighting'}) 

= html + template.render('templates/footer.html', {'links': ''}) 

app = webapp.WSGIApplication([(V, SightinglnputPage)], debug=True) 

def main(): 

run wsgi app(app) 

v\o“t bcmj Imkcd 

if _name_ == '_main_' : 

main() 



Theves only one s»»all 

C. J.L m 




lA/baVs missmj html 

-fy-or* beve? html 



html 

html 
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everything together 



App Engine Code Magnets Solution 

Let's put everything together. Your model code is already in your hfwwgDB . py 
file. You were to move that file into your webapp's top-level folder, as well as copy 
your templates folder in there, too.Your webapp's controller code, in a file called 
hf wwg. py, also needs to exist in your top-level folder. The only problem is that 
some of the code's all over the floor. You were to rearrange the magnets to fix 
things: 


from google.appengine.ext import webapp 

from google.appengine.ext.webapp.util import run wsgi app 

from google.appengine.ext import db 

from google.appengine.ext.webapp import template 

from google.appengine.ext.db import djangoforms ' 1 °'* 


bht data 


wvodel tode- 


import hfwwgDB 


class SightingForm(djangoforms.ModelForm) 


Wse youv model to £reate a sighfmg 
( W that mhevits the 3 
django./WoclelFov-r»" eUss. 

The toncticA handler dlass is talled 
I "Sightmgk*tPa$e" a«d rfc prowdes a 

« JthodYalled V *hi t h res ? 0 ,ds to a 

6{iT web 


class Meta: 


model = hfwwgDB.Sighting | 



class SightinglnputPage(webapp.RequestHandler) 
def get(self): 


html = template.render('templates/header.html', {'title': 'Report a Possible Sighting'}) 
html = html + template.render('templates/form start.html', {}) 


html = html + str (SightingForm()) - /hdlude the ge«ev-ated (, 


°' rn ' ih the HTML v-espohse. 


html = html + template.render('templates/form_end.html’, {'sub_title': 'Submit Sighting'}) 

html = html + template.render('templates/footer.html', {'links': ''}) 



v-esponse bafck to tne wd 
o£ dodc docs just ibat 


app = webapp.WSGIApplication([('/.*', SightinglnputPage)], debug=True) 

def main(): 

run wsgi app (app) 


if _name_ 

main() 


marn 


372 Chapter 10 
























scaling your webapp 



lt's been a long time coming, but you are now ready to test the first version of your sightings form. 

If you haverYt done so already, create an app . yaml file, too. Set the application line to hfwwg 
and the script line to hfwwg. py. One final step is to use the Add Existing Application menu option 
within the GAE Launcher to select your top-level folder as the location of your webapp. 






WAftNIt 


GoogleAppEngineLauncher 

£ 




Rir Stop Browse Logs SDK Condole 


Edit 


Deploy Dashboard 


Nani e 


Path 


mygaetest 


3 


/ U 5 e rs/barryp/He ad Fi rs t Pyt hion / di apte r 1O/rnyg aete st 


/Users/barryp/HeadFirstPython/chapterlO/hfww 


And hevVs youv 

ywahed HTML 

in ali iis glov-y. 



The launeher adds youv- 
webapp inio iis list 
and assigns ii ih e nexi 
available pvoiodol po>-i_ 
in ibis case, 9091. 




Report a Possibile Sighting 


f s http://localhost:SDSl/ 


E (<*_ Coogle 




□Q sss: Apple Yahoo! Coogle Maps YouTube Wlkipcdia Popular 


Report a Possible Sighting 





Name: 
Email: 
Date: 
Time: 
Location: 
Fin type: 
Whale type: 
Blow type: 
Wave type: 


Submit Sighting 


A 


This is looking good. Let’s get a quick opinion from the 
folks over at the HFWWG. 
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make it pretty 


0 



OK, we get it. Web design is not your thing. 

Not to worry, you know all about code reuse, right? So, let 5 s reuse 
someone else’s cascading style sheets (GSS) to help with the “look” 
of your generated HTML form. 

But who can you “borrow” from and not lose sleep feeling guilty over 
it? 

As luck would have it, the authors of Head First HTML with CSS & 
XHTML created a bunch of stylesheets for their web pages and have 
made them available to you. Grab a slightly amended copy of some 
of their great stylesheets from this book 5 s support website. When you 
unzip the archive, a folder called static appears: pop this entire 
folder into your webapp 5 s top-level folder. 

There 5 s a file in static called f avicon . ico. Move it into your 
top-level folder. 


Improve the look of your form 


To integrate the stylesheets into your webapp, add two link tags to your 
header . html template within your templates folder. Here 5 s what the 
tags need to look like: 



Add thcsc -two 1‘mcs io Ihe 

Y*** Vadev.hW' 
icmplaie. 


<link type="text/css" rel="stylesheet" href="/static/hfwwg.css" /> 

<link type="text/css" rel="stylesheet" href="/static/styledform.css" /> 


GAE is smart enough to optimize the delivery of static content—that is, 
content that does not need to be generated by code. Your GSS files are static 
and are in your static folder. All you need to do is teli GAE about them to 
enable optimization. Do this by adding the following lines to the handers 
section of your app . yaml file: 


Swrtd' oy\ “the 


FW.de ihe URL locatio* 
your static Content- 



uri: /static 
static dir: static 
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With your stylesheets in place and your app . yaml file amended, ask your browser to reload your form. 


Lookm) 500d 



Report a Possible Sighting 


( * http://localhost: 8081 / 


C Q/ Coogle 


CO :::: Apple Yahoo! Coogle Maps YouTube 


» 


eport a Possible Sighting 

Name: 



Email: 



Date: 



Time: 



Location: 


Fin type: 



Whale type: 



Blow type: 



Wave type: 




(Submit Sighting j 


A 
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list of choices 


Rcstrict input by providing options 

At the moment, your form accepts anything in the Fin, Whale, Blow, and 
Wave input areas. The paper form restricts the data that can be provided for 
each of these values. Your HTML form should, too. 


«Md First Whale Watahing Group 
C»M , a1 5 ,g ftl , n9 rann » 





Anything you can do to cut down on 
input errors is a good thing. As the youngest 
member of the group, I was "volunteered" to 
work on data clean-up duties... 


Ftr l TjfPf: 

FaotUi 

TriinguOf 

R>jrvd«j 

Whgi* r. w; 


Orca 

Fn 

«Uufl Kfer 


TdI 

tk«J7y 

Om» 

Tip»; 

Flat 

Lsrjo 

Sma* 

6fWi»a 

Woderniu 

HQh 


Providing a list of choices restricts what users can input. 

Instead of using HTML’s INPUT tag for all of your form fields, you can use the 
SELECT/OPTION tag pairing to restrict what’s accepted as valid data for any of the 
fields on your form. To do this, youTl need more HTML markup. Thats the bad 


news. 


The good news is that the form validation framework can generate the HTML 
markup you need for you. All you have to provide is the list of data items to use as 
an argument called choices when defining your property in your model code. You 
can also indicate when multiple lines of input are acceptable using the multi line 
argument to a property. 

Apply these changes to your model code in the hf wwgDB . py file. 


tVtme you*- lists ot values 
hea ' r -the ioy of you*- tode- 


This 

donvention 

helps iden-ti-fy 
-these lists & 

fcoh-tainin^ 
eons-tan-t values. 


FINS = ['Falcate', 'Triangular ', 'Rounded'] 

WHALES = ['Humpback', 'Orca', 'Blue', 'Killer', 'Beluga', 'Fin', 'Gray', 'Sperm'] 
BLOWS = ['Tali', 'Bushy', 'Dense'] 

WAVES = ['Flat ', ' Small', 'Moderate', 'Large ', 'Breaking ', 'High'] 


e- 


location = db. StringProperty (inultiline=True) 

fin_type = db. StringProperty (choices=__FINS) 
whale_type = db.StringProperty(choices=_WHALES) 
blow_type = db.StringProperty(choices=_BLOWS) 
wave_type = db.StringProperty(choices=_WAVES) 


•Swrteh oh mul-tipl 
line ihpu-fc. 


Use youv lisfs 
0-f values when 
de-f inin^ youv 
^vo^ev-ties- 
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scaling your webapp 



With these changes applied to your model code, refresh your web browser once more. 


YouV" 'foVW' IS not OTkVj 
|ookm<\ ^ood) bu*t s 
move viav\< 



The 'loea-fcion” (\c\d 
•s how dispbycd ovev 
multiple lines. 


f* O 


Report a Possible Sighting 




http://localhost: 8081 / 


C (Q t Coogle 


DQ :::: Apple Yahoo! Coogle Maps YouTube 


Report a Possible Sighting 



Name: Eagle-eyedjackjones 


Email: RQj@hfwwg.org 


Date: jan e. 2011 


Time: 802am 


LOCatiOfi: ijust off the coast - really 


close to shore. 


/A 


» 

» 


Fin type: Triangular 


Whale type: Fin 


Blow type: Bushy 


Wave type: Breaking 



Eadh of the "type" 
•fields how have 
drop-down seledtio» 
►«enus assodiated 

with therw. 


Your form now looks great! Go ahead and enter some test data, and then 
press the Submit Sighting button. 


What happens? 
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checking log console 


Meet the "blank screen of death 


jj 


Submitting your form 5 s data to the GAE web server produces a blank screen. 


W/VioofS-./fchdts ttot 

e^adtlY usar—friendlY* 


& ^ A http://localhost:8081/ 

+ 

0 http://local host:8081/ C 

(Q^ Google 

)» 

CQ 

:::: Apple Yahoo! Google Maps 

YouTube 

» 


To work out what happened (or what didn’t 
happen), you need to look at the logging 
information for your GAE webapp. 

If you are running GAE on Linux, your logging 
messages are displayed on screen. If you are on 
Windows or Mac OS X, click the Logs button 
within the Launcher to open up the Log Console 
for your webapp. 


r> n 



Name 

• 

mygaetest 

- 

ESSHS 



& 


Rur Stop Browse 


CoogleAppEngineLauncher 


Q 


Edit 


Deploy Dashboard 


/Users/barrypkHeadFirstPython/chapterlO/mygaetest 


/Users/barryp/ 'eadFirstPython/chapterlO/hfww 


Port 

8080 


8081 


& rs 


Log Console (hfwwg) 


MI 

Ciear 

*** Running dev_app 
--admin_console 
Python command; /us 
WARNING 2010-10-04 
YBOQjqltGGqNBwt Hylle 
WARNING 2010-10-04 
'' PIL 11 module. Impor 


Qt Search llogs 


Search 


INFO 2010-10-04 
INFO 2010-10-04 
INFO 2010-10-04 
INFO 2010-10-04 
INFO 2010-10-04 


server with the following Flags: 

_server™ --port-8081 
r/bi n/python2 .5 

08: 41: 01, 662 datastore_file_stub. py:657] Could not read datastore dato from /var/Folders/Y3/ 
JWk+++TI/-Tmp-/dev_appserver. datastore 

08:41:01,665 dev_appserver. py:3637] Could not initialize images API;; you ore likely missing 
tError: No module named _imaging 

08:41:01,717 dev_appserver_main. py :431] Running Application hFwwg on port 8081: http;//local 

08:41:24,337 dev.appserver. py: 3273] "GET / HTTP/1. 1" 200 - 

08:41:24,359 dev_appserver, py: 3275] "GET /static/styledForm. css HTTP/1.1" 200 - 

08:41:24,374 dev_appserver. py: 3275] "GET /static/hfwwg , css HTTP/1. 1" 200 - 

08:44:27,423 dev.appserver. py: 3275] "POST / HTTP/ 1.1" 405 - 




the Python 
host;8081 



A 


Your request resulted in a 4 05 status code from the web server. According to 
the official HTTP RFG standards document, 4 05 stands for: 

“Method Not Allozved. The method specified in the Request-Line is not allowed 
for the resource identified by the Request-URI. The response MUSTinclude an Allow 
header containing a list of valid methods for the requested resource 33 . 



Vouir las-fc web v-e^ues-fc 
resulted i h a 


Urwmw . tbat s ^s deav* 
as mud) 
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scaling your webapp 


Process the POST withiw your webapp 

What the 4 05 status code actually telis you is that posted data arrived at your 
webapp intact, but that your webapp does not have any way of processing it. 
There 5 s a method missing. 

Take a quick look back at your code: the only method currently defined is 
called get () . This method is invoked whenever a GET web request arrives 
at your webapp and, as you know, it displays your sightings form. 

In order to process posted data, you need to define another method. 
Specifically, you need to add a new method called post () to your 
SightinglnputPage class. 


App Engine handles requests as well as responses 

Your get () method produces your HTML form and returns a web response 
to the waiting web browser using the self. response object and by 
invoking the out.write () method on it. 

In additon to helping you with your web responses, GAE also helps you 
process your web requests using the self. request object. Here are a few 
lines of code that displays all of the data posted to your web server: 




pet me a 

„ethod talled 

vt 



7 n * to use sel+ with 

all youv me-fchods. 


' ar y»r*eivts<./ meth 
a ■* Held h^naes used 


def post(self): 

for field in self.request.arguments(): 
self.response.out.write(field) 
self.response.out.write(': ') 

self.response.out.write(self.request.get(field)) 
self.response.out.write('<br />') 



re-tun-Ks a lis-fc 
youv form. 


The “ytO" wcthod vetuv»>s the value assodiated 
with the yrovided tovm tield «a*»>e- 


So.. .if you know the name of your form field, you can access its value from 
within your webapp using the self . request. get () method. 


But what do you do with the data once you have it? 
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storing data 


Put your data m the datastore 

Your data is sent to your webapp by GAE and you can use the s e 1 f . 
re que st. get () method to access each input field value by name. Recall 
the BirthDetails model from earlier in this chapter: 


TWis todc is '«* a -file 
talled 


— 

from google.appengine.ext import db 

class BirthDetails(db.Model): 

name = db.StringProperty() 

date_of_birth = db.DateProperty() 
time_of_birth = db.TimeProperty() 


Assume that an HTML form has sent data to your webapp. The data is 
destined to be stored in the GAE datastore. Here’s some code to do the heavy 
lifting: 


HTML vesyonse 

{fi say ^hanks/ 

Schd your 
response io -fche 
warfcihj) web 
browser. 


def post(self): 

new birth = birthDB.BirthDetails() 


Crcaie a „ew "Biv&De-tails" 

objedt -to hold yo«v data. 


new_birth.name = self.request.get('name') 
new_birth.date = self.request.get('date_of_birth') 
new_birth.time = self.request.get('time_of_birth')) 


new_birth.put() 


Put (save) your data to the <$AE 
datastore- 


£jet eath ot the 
-fovw\ s data valucs 
and assiy* tbcw\ to 
your new objett s 
attvibutcs. 


: template.render('templates/header.html' , {'title': 'Thank you!'}) 

: html + "<p>Thank you for providing your birth details.</p>" 

= html + template.render('templates/footer.html', 

{'links': 'Enter <a href="/">another birth</a>.'}) 

self.response.out.write(html) 


There’s nothing to it: create a new object from your data model, get the 
data from your HTML form, assign it to the object 5 s attributes, and then use 
the put () method to save your data in the datastore. 
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scaling your webapp 



Based on what you know about how to put your HTML fornTs data into the GAE datastore, 
create the code for the post () method that your webapp now needs. Some of the code has 
been done for you already. You are to provide the rest. 


def post (self) : 


Pu-t youv todc 
heve- 


html = template.render('templates/header.html’, 

{'title': 'Thank you!'}) 

html = html + "<p>Thank you for providing your sighting data.</p>" 
html = html + template.render('templates/footer.html', 

{'links': 'Enter <a href="/">another sighting</a>.'}) 

self.response.out.write(html) 
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post to datastore 



ExwctSe 


§0(.|#tl0H 


Based on what you know about how to put your HTML fornTs data into the GAE datastore, you 
were to create the code for the post () method that your webapp now needs. Some of the 
code has been done for you already. You were to provide the rest. 


def post(self): 

— IvPv/v/ 


Crcaic a "Sigh-Ug" ob j edt 



— scl-f.\rc^ucs*t 

y\cv/_=■ scl-f.v-c^ucs-t^ct^CmailO 



r\cy/_sigh*t'm^.da*tc — sel^.v-e^uest.^etOdateO 
r\ew_si^htm3*tirne — sel-Pv-e^uest ~\ 

hcy/^sightm^lodatioh ==■ sel^.v-e^ueskgetOodatioir/) 
ne w__si (jbtin*)- -P i n__ty pe — sel-Pv-e^uest 3 yfc 0 

hcy/^sightmg.y/halc^-typc — sel£v-e^uest.^etCy/hale__*typeO 
new^sightinjblow^type —sel^v-e^uest-jet^low^typeO 
r\ew_si^btm$wave_*type — sel£v-e^uest-^et^wave^jtypeO 



Fov ea£h of the j^ta 
valucs \re£eived -Pvor* 

HTML &*•*, 

assi 9 h ther» to -the 
attvibutes of the 
»ewly Oeated objeci 


. . , i• . in d' ” SWe 70 WV ? 0 ? «laied objeti m 

»evj_*$hh»wtU - daias U e . 


html = template.render('templates/header.html’, 

{'title': 'Thank you!'}) 

html = html + M <p>Thank you for providing your sighting data.</p> M 
html = html + template.render('templates/footer.html', 

{'links': 'Enter <a href= M / M >another sighting</a>.'}) 


self.response.out.write(html) 
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scaling your webapp 



'esT DriVq 


Add your post () code to your webapp (within the hfwwg. py file) and press the Back button on 
your web browser. Click the Submit Sighting button once more and see what happens this time. 

tteves your 
-('ovw' wrtVi “tVic 
data yiavlmj to 

bc suWrblcd- I ^ ^ ^ 


Report a Possible Sighting 


t 


0 http://localhost:8081/ C| (O? Google 


m 


Apple Yahoo! Google Maps YouTube 


» 

» 


Report a Possible Sighting 


Namei Eagle-eyedjackjones 


Email: Wj@hfwwg.org 


Date: jan 6 ,2011 


But \wh em you click 
"the button, somethmg 
bad has happe«ed...you^ 
webapp has cv-ashed. 


Time: 8 02am 


Lg 

0 D O 


httpi//localhost:8081/ 




+ ‘ N h ttp : / / Io c al hos t : S 0 S 1 / 


C (O? Google 

F 

□P ”” 

Apple Yahoo! Google Maps 

YouTiibe Wikipedia PopyfarT 



Wha 

Blo 

Wav 


Traccback fnost recent call last): 

File " /Applications/CoogleAppEngineLauncher.app/Contents/Resources/CoogleAppEngine-de 
handler . post { ^groups} 

File " /Users/barryp/fleadFirstPython/chapterlO/hfwwg/hfwwg.py ", line 27, in post 
new_sighting.date ■ self.request*getdate ’} 

File " /Applications/CoogleAppEngineLauncher.app/Contents/Resources/CoogleAppEngine-de 
value ■ self.validate^value) 

File " /Applications/CoogleAppEngineLauncher.app/Contents/Resources/CoogleAppEngine-de 
value = super { DateProperty, self).validate!value) 

File " /Applications/CoogleAppEngineLauncher.app/Contents/Resources/CoogleAppEngine-de 

{ self . r.ane, self .data_type ._nane_ } } 

SadValueErrors Property date nust be a date 


t 


It looks like you have a ? <roble«. with the 

tovn>at ci youv date ro ? erty, doe »t «t. 


3 


^ ► A 


Phooey.-.thafs disappointing, isn’t it? 

At the very least, you were expecting the data from the form to make it into 
the datastore.. .but something has stopped this from happening. What do you 
think is the problem? 


you are here ► 
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conservative responses to liberal requests 


VoWi break the "robustwess priwciple" 

The Robustness Principle States: u Be conservative in whatyou send; be liberal 
in whatyou accept. ” In other words, don’t be too picky when requesting data of 
a certain type from your users, but when providing data, give ’em exactly what 
they need. 

If you make it too hard for your users to enter data into your system, things 
will likely things break. For instance, within your model code, consider how 
date and time are deflned: 


A date, a»d NOTHI 
tat a date 'will do. 



date = db.DateProperty() 
time = db.TimeProperty() 


The trouble is, when it comes to dates and times, there are lots of ways to 
specify values. 



You r*us-t pvovide a 
valid value -Pov -time. 
A^y-thihg else is simply 

WWACCEPTABLE. 
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scaling your webapp 


Acccpt almost any date and time 

If you are going to insist on asking your users to provide a properly formatted 
date and time, you’11 need to do one of two things: 

• Specify in detail the format in which you expect the data. 

• Convert the entered data into a format with which you can work. 

Both appoaches have problems. 

For example, if you are too picky in requesting a date in a particular format, 
you’11 slow down your user and might end up picking a date format that is 
foreign to them, resulting in confusion. 

If you try to convert any date or time entered into a common format that 
the datastore understands, you’11 be biting off more than you can chew. As 
an example of the complexity that can occur, how do you know if your user 
entered a date in mm/dd/ yyyy or dd/mm/yyyy format? (You don’t.) 


There is a third optiow 

If your application doesn’t require exact dates and times, dordt require them of 
your user. 

With your sightings webapp, the date and time can be free-format fields that 
accept any value (in any format). What’s important is the recording of the sightings 
not the exact date/time it occurred. 



0( doursc; oihev- webdpps 
not be as tast and 
loose wi-th dates a»d times. 
Men tba-ts the case, youll 
need to revert one ot the 
options disdussed eavliev on 
this page ahd do -the bes-fc 


you £a*. 


Use "db.StringPropertyO" for dates and times 


If you relax the datatype restrictions on the date and time fields, not only 
do you make is easier on your user, but you also make it easier on you. 

For the sightings webapp, the solution is to change the property type for 
date and time within the hfwwgDB . py file from what they currently are 
to db.StringProperty() . 


date = db.StringProperty() 
time = db.StringProperty() 



|t’s a smdll than^e, 
but it'll n>ake ali the 

dittevende- 


Let’s see what difference this change makes. 


you are here ► 
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te st drive 



'esT DRive 


Change the types of date and time within htwwgDB . py to db . StringProperty (), being 
sure to save the file once youVe made your edit. Click Back in your web brwoser and submit your 
.folks... sightings data once more. 

Irt’s b"i 

SwdCess/ H appeav-s io 
have wov-ked -this ti* e . 

I 

Thank you! 

C Ipy Google » 


f* ^ ' r> Report a Possible Sighting 


+ 

0 http://localhost:8081/ C fQ» Google 

)» 

m 

:::: Apple Yahoo! Google Maps YouTube 

» 


Report a Possible Sighting 

Name: ELagle-eyed JackJones 

Email: Wj@hfwwg.org 


* ^ ^ 


Date: jan 6 ,2011 
Time: 8 02 am 


LOCation: fjjust off the coast - really 
ciose to stiore. 

Fin type: Triangular ~T~) 

_X' 


http://local host:8081/ 


□Q :::: Apple Yahoo! Google Maps YouTube 


» 


Thank you! 

Thank you for providing your sighting data. 
Enter another sighting. 

f* rs rs 


j^st b> bc 


sure 


1 


Report a Possible Sighting 

C| (Google 


0 http://localhost:8081/ 


1 

J 


» 


DQ :::: Apple Yahoo! Google Maps YouTube 


» 


Whale type 
Blow type 


Bushy 


m 


Wave type: Breaking i 

^Submit Sighting) 


Report a Possible Sighting 


Name: 

Email: 


Mary KtliflhtY 


Hiky@hfwwg.org 


Date: Wed.Jan 12 , 2011 


Time: 9 43am 
Location: 


Off Lighthouse Rock, looking 
north-east. 


By relaxing the restrictions you placed on the types of data 
youil accept, your webapp now appears to be working fine. Go 
ahead and enter a few sightings by clicking on the link on your 
thank-you page and entering more data. 


Fin type: Rounded ~T") 

Whale type: oca _ b 

Blow type: Dense C ' 

Wave type ; Moderate i 

( Submit Sighting ) 


A 
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scaling your webapp 


With a few sightings entered, let's use App Engine's included developer console to confirm that the 
sightings are in the datastore. 

To access the console, enter http : //localhost: 8081/_ah/admin into your web browser's 
location bar and click on the List Entities button to see your data. 



T ftg k Q gftgft? 


Oror. Jofrs 

Key ID 

Key Nsm* blowtype 

date 

email 

fln_type 

location 

name 

time 

wavetype 

whaletype 

X.MPP 

aaVoZnC3.. 6 

Bushy 

Jan 5, 2011 

eejgihfwwg org 

Tnangular 

Just off the coast - 
really dose to shtxe. 

Eagle-eyed 
Jack Jones 

8 02am 

Breaking 

Fin 

IrUAind Mail 

Q agVo7nd3 7 


Wnd. J.m 

12. 2011 

mky@hfwwg nrg 

Rntmrtnd 

Off i tghthour.fi Ror.k, 
looklng north-nast 

M.vy Knlghty 

9 43.*im 

Moriomto 

Or.n 


ayVoZr>d3 , . 8 

Tali 

Fetxuary 20. 
2011 

com 

Falcata 

Lookiny wwt at 
Smuygler‘s Cuvo 

James Pag» 

Earty. at 
dawr. 

Moderatu 

Gmv 



Buahy 

2/Marf2011 

rob@zep com 

RoorxJed 

M>sty Mountain Bay. 

Bob Piant 

4 4Spm 

Flat 

Humpbeck 


1 



Your GAE webapp is now ready for prime time. 

Before you deploy it to Google 5 s cloud infrastructure, let’s run it by the folk at 
HFWWG to see if they are happy for their webapp to “go live.” 


you are here ► 
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restrict to registered users 


It looks like youYc wot quite dowe yet 




Is this a big change? 

You would imagine that it would be. You’11 have to create an new entity to 
hold your registered user login information, and you’11 also need another form 
to ask users to provide their registration data (which you’11 need to store in the 
datastore). With that in place, you’11 need yet another form to ask your users to log 
in, and then you’11 have to come up with a mechanism to restrict only registered 
and logged-in users to view your webapp’s pages, assuming you can come up 
with something robust that will work...? 

Or...as this is GAE, you could just switch on authorization. 
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scaling your webapp 


Sowetiwes, the tiniest change can make 
ali the difference... 

The engineers at Google designed App Engine to deploy on Google 5 s cloud 
infrastructure. As such, they decided to allow webapps running on GAE to 
access the Google Accounts system. 

By switching on authorization, you can require users of your webapp to 
log into their Google account before they see your webapp 5 s pages. If a user 
tries to access your webapp and he isn’t not logged in, GAE redirects to the 
Google Accounts login and registration page. Then, after a successful login, 
GAE returns the user to your waiting webapp. How cool is that? 

To switch on authorization, make one small change to your app . yaml file: 


s <all "thevr 
is io i-fc. 


Now, when you try to access your webapp, you are asked to log in before proceeding. 


— 

application: hfwwgapp 
version: 1 
runtime: python 
api_version: 1 

handlers: 

- uri: /static 
static_dir: static 

- uri: /.* 


script: hfwwg.py 



TViis is Viovj the l<*y« 
Stveen loohs Wrthrn the 
test envivonment 

cm ^ ton>?i*tev. 


1 9 © O Login 


+ 0 http://localhost:8081/_ah/loc C (O? Google 

. • 

□Q :::: Apple Yahoo! Google Maps YouTube Wikipedia » 




Not logged in 



Email: test@example.com 

□ Sign in as Administrator 


Login Logout 


,:;v 


you are here ► 
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log login info 


Capture your u$er'$ Coogle IP, too 


Now that your webapp requires your users to log in, let’s arrange to capture 
the user login information as part of the sighting. 

Start by adding the following property to your entity 5 s list of attributes in your 
hf wwgDB . py file. Add it right after the wave_type property 

Create a *ew 
attribute m 70UV 
“Si^htin^ dlass... 



•. ahd set its 
property typ c . 


Let’s ensure that Django’s form validation framework excludes this new attribute 
when generating your HTML form. Within your hf wwg . py file, change your 
Sighting Form class to look like this: 


Make suV " c Pja*<y> 
doesnt mdlude 


the ryt* attribute 
'm your (jenerated 
-form- 



Staying within your hf wwg . py file, add another import statement near the 
top of your program: 



/wfov-t <$AB‘s 
^oogle Addounts 

API. 


In your post () method, right before you put your new sighting to the 
datastore, add this line of code: 



Every time a user adds a sighting to the datastore, GAE ensures that the user 5 s 
Google Account ID is saved, too. This extra identification information allows the 
HFWWG to track exactly who reported which sighting, and should (hopefully) cut 
down on the amount of spam your webapp might attract. 


Whe* you put your 

data -to ite datastore, 
this dode indludes 
the £joo$le IP ot the 

durrertly lo$ed-'m user. 


AII that’s left to do is to deploy your webapp to Google’s cloud. 
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scaling your webapp 


Peploy your webapp to froogle s cloud 

With your webapp developed and tested locally, you are now ready to deploy 
to the Google cloud. This is a two-step process: register and upload. 

To register your webapp on the Google cloud, click the Dashboard button 
on the GAE Launcher. 


The Dashboard" button oj>ens 
your u/eb browser and takes you 
to the $AB "/l/Jy Application" 
page (a+ter you sign j h 
your google |D). 


r> n 


m 


» ^ ■ mi; # 

Rur Stop Browse Logs SDK Console Edit 


GoogleAppEngineLauncher 


Q 



Deploy Dashboard 


Name 

mygaetest 


Path 


/Users/barryp/HeadFirstPython/chapterlO/mygaetest 


/Users/barryp/HeadFirstPython/chapterlO/hf 


Port 

8080 


8081 


<r 


B)y\ L-i nuv-, use the 

"appttg pystript to 
deploy- 


^ ^ Create an Application 

(e) CD ® QT http://appengine.google.com/start/createapp 


^ T 1 SS3' seventh inning stretch 


CD 

Q/ 


1 


BBC News Si RTt News C-Mail 0'Reilly Radar A Head First Labs 5* LJ - Front S\ LJ - News Safari CNews Ireland Si Twitter TT: 2010/2011 

C± 


Create an Application 


Google app engine 

Create an Application 

You have 7 applications remaining. 

Application Identifier: 

.appspot.com Check Availability 

You can map this applicatiori to your own domain later. Leam more 

Application Title: 

I <- 

Displayed when users access your application. 


I Mv Account | Help | Sion out 



Enter your v/ebapps na«>e i 
the box, then click on the 

Chedk Availability” button 



Pont use "htwuigapp” as 
thats already taken- © 


Authentication Options (Advanced): Leam more 

Google App Engine provides an API for authenticating your users, including Google Accounts\Google Apps and OponlD If you choose to use this feature for some parts of your site. 
you'll need to specify now what type of users can sign in to your application: 

Open to ali Google Accounts users (default) 

If your application uses authentication, anyone with a validGoogle Account may sign in. (This inclutas all Gmail Accounts.but does *not* include accounts on any Google Apps domains.) 
Edit 


Create Application Cancel 


Optionally, enter a title tor ^our -ebapp, and then 
tlitk the "Create ApplitaW button- 



© 2008 Gooale 1 Terms of Service 1 Privacv Policv 1 Bloa 1 Discussion Forums 


Do ne 


/fi 


Assuming all went according to plan and GAE confirmed that your 
application has been created, all that 5 s left to do is to deploy. Return to the 
GAE Launcher and click on the Deploy button. The console displays a 
bunch of status message while the deployment progresses. If all is well, you’11 
be told that “appcfg.py has finished with exit code 0”. 


Your GAE webapp is now ready to run on Google’s cloud. 


you are here ► 
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te st drive 



Let's take your webapp for a spin on Google's cloud. Open your web browser and surf to a web 
address that starts with your webapp's name and ends in .appspot.com. For the HFWWG webapp, the 
web address is http://hfwwgapp.appspot.com. When you first attempt to go to their webapp, App 
Engine redirects you to the Google login page. 




Google Accounts 



C e) 


S I N 


it ) ( 


google.com https://www.goog 'fi 


Google 




BBC News RT£ News G-Mail 0’Reilly Radar Head First Labs LJ - Front 3* LJ - News Ks Safari GNews Ireland » 

■ +1 


Li! 


Google Accounts 


Google accounts 

HeadFirst Whale Watching Group uses Google Accounts 
for Sign In. 

Google is not associated with the contents of HeadFirst Whale Watching Group or its 
owners. If you sign in, Google will share your email address with HeadFirst Whale 
Watching Group, but not your password or any other personat information. 

HeadFirst Whale Watching Group may use your email address to personalise your 
experience on their website. 


provide 7owr $00^1 e IP and 
Dassxovd, or siy. * *or a nev/ 
£joo$le atdowt (i-r Y ow c ' ori ^ 
Kave one alveady)- 



Sign in with your 

Google Account 

o 

Email: 

e.g. pat@example.com 

Password: 

(3 Stay signed in 
Sign in 

Can't acoess vour account? 



©2010 Google - Google Home - Terms of Service - Privacv Policv - Help 


Don't have a Google Account? 
Create an account now 


Done 


e 


/A 
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After a successful login, your sighting form appears. Go ahead and enter some test data: 




Report a Possible Sighting 

«E- ...e, X' (£) Oi/frO (a 


CD 
■' CoQ 


The si^hti*^ £orm served 
fvom the ^oo^le tlowd ' s 
E/ACTL-Y' the same as the 
•form served by y°'* lr 't es ^ 

SCWCV. 


BBC 

Lk 


News RTt News C-Mail 0’Reilly Radar 
Report a Possible Sighting + 


* r> 


< }_ G *_) ' v 1 #) C-' ^g oogle.com https: 


BBC News Sv RTt News C-Mail 0'Reilly Radar Head F 
Data Viewer - HeadFirst Whale Wa... + 


Report a Possible Sighting 

Name: Jonathon Baldwin 

Email: 



jpj@zep.com 


Date: Mar 3 ,2011 
Time: 


ll:59am 


I nr^tinn- At Bluffer's Beach, 

‘ about 50m from shore. 


• 7 ^ ^ b ' nh > ^ £or,sole. The Uf 

l ? l,Ule V»* the test do»sole, 

but you da« use the Datastore l/iewer to 

doh+,rm that your data has beer, stored 
Corredtly. 


Google app engine 


Fin type: 
Whale type: 
Blow type: 
Wave type: 


Triangular 


Cray 


T 


Tali 


1] 


Moderate 


hfwwgapp Version: 1 

Main 

Dashboard 
Quota Details 

Loqs 

Cron Jobs 
Task Queues 

Blacklist 

Data 

Datastore Indexes 


Sjbm t Sighting 



« Mv Applications 


Query 


Create 



By kind: Sighting kin^= Done 

ffl Qotions 


_ A 


Sighting Entities 




Datastore Viewer x 


Datastore Statistics 

Blob Viewer 

Administration 
Application Settinqs 

Permissions 

Versions 
Admin Loqs 


0 1-1 

□ ID/Name blow_type 

date 

email 

fin_type 

location 

name 

time 

wave_type 

whale_type 

which_user 

□ id=1 Tali 

Mar 3, 
2011 

jpj@zep.com 

Triangular 

At Bluffer*s 
Beach, about 
50m from 
shore. 

Jonathon 

Baldwin 

11:59am 

Moderate 

Gray 

hfpython 


w 


Delete 



Cli^k Oh -this lihk io scc 
youtr da-fca as sWcd ih 
thc ^ooglc doud. 



1-1 


Done 


fi 


you are here ► 
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a winning webapp 


Your HFWWft webapp is deployed! 



O 

o 



This is professional work. 


O 


This is waaaay cool. Look 
at that fabulous webapp. 
This is exactly what we 
need. Super work! 



O 


YouVe built a great data-entry webapp and deployed it on Google’s cloud. No 
matter how busy things get, whether there’s a handful of sightings per day 
or tens of thousands ,your webapp can handle the load , thanks to Google 5 s App 
Engine. And, best of all, the cash-strapped HFWWG doesn’t pay a penny 
until their sightings activity reaches the level of millions of sightings per 
month! 


p-,a you evcrk t^at 

you w verte all 7°^ 

dode usmj Python 2-5? 
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scaling your webapp 



Your Pythow Toolbox 

You’ve got Chapter 10 under your 
belt and you’ve added more great 
Python technology to your ever- 
expanding Python toolbox. 


* "patasUe" - ^ data ttSed 

by ^oo^le A?? Y crn '* r ' cr '™'1 

s*bov“C Y ow 

9 - u* rtawe wsed ^ a ' Vo ' w 

data”. 

# «pyo^vt/ - tKe »a»»e used U a "data 
vaW”. 


■ Every App Engine webapp must have a 
configuration file called app. yaml. 

■ Use the GAE Launcher to start, stop, 
monitor, test, upload, and deploy your 
webapps. 

■ App Engine’s templating technology 
is based on the one use in the Django 
Project. 

■ App Engine can also use Django’s Form 
Validation Framework. 

■ Use the self. response object to 
construet a GAE web response. 

■ Use the self. request object to 
access form data within a GAE webapp. 

■ When responding to a GET request, 
implement the required functionality in a 
get () method. 

■ When responding to a POST request, 
implement the required functionality in a 
post () method. 

■ Store data in the App Engine datastore 
using the put () method. 


you are here ► 
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11 dealing Wlth complexit/ 



lt’s great when you can apply Python to a specific domain area. 

Whether it’s web development, database management, or mobile apps, Python helps you 
get the job done by not getting in the way of you coding your solution. And then there’s 
the other types of problems: the ones you can’t categorize or attach to a domain. Problems 
that are in themselves so unique you have to look at them in a different, highly specific way. 
Creating bespoke Software Solutions to these type of problems is an area where Python 
excels. In this, your final chapter, you’ll stretch your Python skills to the limit and solve 
problems along the way. 


this is a new chapter 
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pace data 


Whaf $ a good time goal for the 
wext race? 


The Head First Marathon Club has spent years collecting and 
collating data on their distance runners. Over time, this 
data has helped the club produce a large spreadsheet of pace 
data that helps their runners predict their performance over 
various distances. The spreadsheet is huge and runs to 50 
columns of tightly packed data. 

Let 5 s take a look at the club 5 s data, as well as the way the 
runners and their coach use it. 


iWs a portio* of IV* MaratU» 
CWb’s spreadsheet data- 



Vr 



The 'tiwved 
dista Ut is 


The fvcdifrted 
w>avathoy\ ^oal 


File tdit Vie*’ In-ier Format Forni Ioda Fleo 

li $ % 123« 1 Dpi r 


j 


rm. 


^ r St Q 


■V E 


Formula: VU2 


4 

j 

a 

s 


h 

a 

c 

0 

E 

i- 


V02 

34.5 

62.9 

61.1 

79.3 

77.5 

75.6 


Ztfl-lr 

a:oo 

3:19 

6.21 

6:33 

6:44 

6:50 

5Tt 

12:49 

13“ftK 

13:24 

13:42 

14:59 

14:19 

541-11 


21.46 

zziTr- 

22:47 

23:15 

23:50 


26.54 

27:39 

26:95 

■^45 


39:94 

15fe 

4l:3l 

42.27 

43:24 

44:23 -^ ( 

45:24* ^ 

45:24 

hunnT 

44.45 

45:46 

45:45 

47:51 


59:92 

awc 

56.29 

57.45 

59:93 

1:99:23 

1:91:45 

1:93:05 

13.imi 

59:49 

1:01:09 

1:02:32 

1:93:56 


1:90:51 

25 ii. 

1:11:43 

1:13:20 

1:14:59 

1:15:40 

1:15:2^ y 

1:20:10 

Mh 

1:27:10 

1:19:05 

1:31:05 

1:33:11 


1:37:25 

-i . 

Marathon 

2:05:34 

2.06:24 

2:11:17 

2:14:15 

2:17:16 \ ■ 

2:20:21 
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dealing with complexity 



This benchmark allows me 
to look up or down the column 
to select a target time for 
any other dis+ance, such as a 
mara+hon, for example. 






spreadshee-fc j s a |,^| e 

* I _ / . | | . , , 



This st 

— ■ ^ v iw n iit ii 

i^imidatih 9 ...but doh'-t wo^y. 
jO\a II soV"t it out sooh. 




G 

H 

i 

j 

K 

L 

M 

H 

o 

7Ar.2 

72.5 

70.9 

59.4 

67.9 

65.4 

54.9 

53.5 


9:03 

9:20 

9:33 

9:45 

9:59 

10:13 

10:26 

19:41 


14:30 

14:5B 

15:16 

15:39 

16.09 

16.22 

15:44 

17:96 


54:22 

24:55 

25:25 

26:03 

26.35 

27:14 

27:51 

25:20 


36:45 

31:25 

32:09 

32:52 

33:35 

34.22 

35:90 

35:55 


47:27 

43:31 

49:35 

50:43 

51:52 

53:92 

54:14 

55:27 


51:09 

52:1 B 

53.29 

54:41 

55:55 

57:11 

50:20 

59:47 


1:114:33 

1:06.00 

1:07.29 

1:09.91 

1:10:34 

1:12:09 

1:13:46 

1:15:26 


1:03 :Zl 

1:09.53 

1:11:20 

1:13.94 

1:14:43 

1:15:24 

1:15:07 

1:19:52 


1:21 :5B 

1:23:49 

1:25.49 

1:27:37 

1:29:33 

1:31:37 

1:33:40 

1:35:47 


1:39:37 

1:41:52 

1:44.09 

1:40.39 

1:40:54 

1:51:21 

1:53:51 

1:55:25 


2:23:31 

2:26.44 

2:30.02 

2:33.25. 

2:36:52 

2.49:24 

2.44:00 

2:47:42 
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rain delay 


$o...whaf $ the problew? 



At the moment, we print our data onto 
multiple sheets of paper, which we carry with us. 
Most of the time, this works fine. But when it rains 
or gets really windy, our pages are either soaked 
through or they end up all over the place. 


AII -fchese shee-b arc 
a pain...esp C £ially *, h 
tkc \raih. 



V 


Not to mention: forgetting the sheets, keeping the sheets up to date, and 
having to flip back and forth through the sheets looking for a closest match. 

Of course, word of your newly acquired Python programming skills is 
getting around, especially among the running crowd. Ideally, the Marathon 
Club needs an Android app that can be loaded onto a bunch of phones and 
carried in each coach’s pocket. The app needs to automate the lookup and 
distance predictions. 


Are you up to the challenge? Do you think you can help? 
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dealing with complexity 


Start with the data 

For now, let’s not worry about creating the Android app; you’11 get to that 
soon enough. Instead, let 5 s solve the Central data wrangling problem 
and then, when you have a working solution, we’ll worry about porting your 
solution to Android. We’ll begin by getting the data into a format you can 
easily process with Python. 

Most spreadsheet programs can export data to the widely used GSV format. 
The club has done this for you and created a file called PaceData.cs v, 
which includes the data for each of the rows from the original the spreadsheet. 

Here’s a sample of the raw data from the start of the GSV: 

tv fc* w o* a* j * 

like nuJoevs lou-t ave atWly . . , m ^ wes in eadh dolum»- 

V 






r 


D® thh! 



Grab a copy of 
PaceData . csv from this 
book’s support website. 


V02,84.8,82.9,81.1,79.3,77.5,75.8,74.2,72.5,70.9,69.4,67.9,66.4,64.9,63.5,62.1,60.7,59.4,58.1,56.8,55. 
8:00,8:10,8:21,8:33,8:44,8:56,9:08,9:20,9:33,9:46,9:59,10:13,10:26,10:41,10:55,11:10,11:25,11:40,1 
42:49,13:06,13:24,13:42,14:00,14:19,14:38,14:58,15:18,15:39,16:00,16:22,16:44,17:06,17:30,17:53,18: 





The tinst value o» eadh of the vest 

the lines is the t.r»ed distande 
OV* Irow label. 


The vest of eadh line is a Itet of vedovded 


r tJharpen your pencil 



You somehow have to model the data from the CSV file in your 
Python program. Can you think of a data structure that might 
help here? Justify your selection. 
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multiple associations 


| J \ 

Solutiori 


You somehow have to model the data from the CSV file in your 
Python program. You were to think of a data structure that might 
help here? You were also to justify your selection. 


IX molon ... 
-thev-es lo-ts 
-to iWmk 
abou*t here- 


The lis*t o-f headings dan be s-bored in a L|£T- 

The list o-f times -from eadh row dan also be s*tored in a L|£T, bu*t they 
also need *to be assodiated y/i*th *the headings in *the very -first row o-f 
data, so maybe a VICTIONARY is what*s needed here? 


Maybe it*s some COMBIMATIOM o-f tbe two?(? 


Take awother fook at the data 

The first row of data in the CSV file is the column headings, with the 
very first value on the line, the V02 string, being redundant (it won’t ever be 
used in this version of the app). The rest of the first line’s data are headings 
associated with the time values in each of the columns. 

Of course, the data in the columns is also associated with each row, which is 
identified by a row label in the first column, such as 2mi, 5k, and so on. 

Let 5 s look at the data in the CSV file again, which has been reformatted to 
help highlight the associations. 

The dolumn headings 3re 



But can we capture all these associations in code? 
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dealing with complexity 



What hccds 


Marathon Magnets 

Here's some code that reads the raw data from from the CSV data 
file. The column headings from the first line are loaded into a list 
called column_headings. The rest of the data (all the rows of 
times) are loaded into a dictionary called row_data, with each row 
of data keyed with the row label string from the start of each line. 
Of course, as luck would have it, someone was cleaning the fridge 
door, and theyVe left a bunch of magnets on the floor. See if you 
can arrange the magnets into their correct order. 


io 90 here? 



with open('PaceData.csv') as paces: 


Prodess the 

list here- 


for each line in paces: 


Proeess 

Vow_data w 

here- 



num_cols = len(column_headings) 
print(num_cols, end=' -> ') 
print(column_headings) 

num_2mi = len(row_data['2mi']) 
print(num_2mi, end=' -> ') 
print(row_data[’2mi']) 


num_Marathon = len(row_data['Marathon']) 
print(num_Marathon, end=' -> ') 
print(row data['Marathon']) 



Wrth the data 
loaded, this tode 
lets you chedk i-f if'. 
all OK. 



you are here ► 
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read table data 



Marathon Magnets Solution 

Here's some code that reads the raw data from from the CSV data 
file. The column headings from the first line are loaded into a list 
called column_headings.The rest of the data (all the rows of 
times) are loaded into a dictionary called row_data, with each row 
of data keyed with the row label string from the start of each line. 
Of course, as luck would have it, someone was cleaning the fridge 
door, and theyVe left a bunch of magnets on the floor. You were to 
see if you could arrange the magnets into their correct order. 


row data = U ~| 


You hccd io be 


di£-fciohdv-y 

with open('PaceData.csv') as paces: 


y e t> <*eate an emfty 

+«r the row times. 


Create the 

tolumn headings I -■ ir , . i » n 

fvor» the -first — column_headings = j paces . readline () . stnp () . split( , ) 


rv-om 

ImC o-f da^- 


column_headings.pop(0) 

Delete the Pirst ^ 

heading, the for each_line in paces: 

VOZ^ring-you don't 
need it. 


d R ea( j a line Prom the tle, st"? *t 
^wnwanted whites?ate, and then 
s?lit the line on tomma. 

FVoCess the vest of the -Pile. 


&*brafcfc “tbc 
row label- 


row — each_line.strip().split( 
row label = row.pop(O) 


row data[row label] = row 


! ! 



|t's the same deal here: 
take the line, sfcri? it, and 
-tbcn sflrl ov\ 


num_cols = len(column_headings) 
print(num_cols, end=’ -> 1 ) 
print(column headings) 


Mse -tbc Irow label 
‘toge-ther wi-th the res-t 
ot -the 1‘me's da-fca io 
upda*te the didiiohav-y. 


num_2mi = len(row_data[’2mi’]) 
print(num_2mi, end= 1 -> ’) 
print(row data[’2mi’]) 


num_Marathon = len(row_data[’Marathon’]) 
print(num_Marathon, end=’ -> ’ ) 
print(row data['Marathon 1 ]) 
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dealing with complexity 



Load your code into IDLE and, with the CSV in the same folder as your code, run it to see what you 
get on screen. 


Your £ode 
i» IDLE.'-^ 


^ ^ rcwathort.py - /Users/barryp/HeadFirstPython/chapterll/marathon.py 

row data = {} 

with open( r PaceData,csv 1 ) as paces: 


column headings = paces«readline().strip()«split( 1 f 1 ) 
column headings■pop(0) 


The outfw-t donWs 
■tha-t eath rox o+ 
da-ta has 50 da-ta 


for eachJLine in paces: 

row = eachJLine.strip().split( 1 , 1 ) 
row__label = row,pop(0) 
row" data [ row_Iabel] = row 

num cois = len( column headings) 
print (num cois, end= 1 -> ') 
print( column headings) 


num zmx - len^row 

^ ll fl- i *^T1B| ■T 







That’s a great start: you Ve managed to read the data from the CSV and put 
the headings into a list and the data into a dictionary. 


What’s next? 


you are here ► 
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link data structures 


0 


Did you forget to associate 
each time on each row with its 
heading? At the moment, the 
list and the dictionary are 
disconnected... 



Yes, the two data structures should be linked. 

At the moment, the column_headings list and the row 
data dictionary are not linked in any way, and they need to 
be. What we need is some way to connect each of the times in 
each row with the heading that tops their column of data. 

What options do you have here? 

When it comes to linking (or associating) two data items with 
each other, the Python dictionary is the data strucutre of 
choice, isrft it? 
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dealing with complexity 


Store each time as a dictionary 


Rather than simply storing each time in the row data dictionary as a 
number, let 5 s store the data as as a dictionary , with the key set to the time and 
the value set to the column heading. That way, you can quickly and easily 
determine for any time which column it is associated with, right? 


Here 5 s a portion of what the data structure looks like in Python’s memory 
once this association exists: 

The u vow_data 

di£.tio*avy '°' fi 5 ev ‘ 

towtains 3 list 





{ 


f»skad ot a single i lrne va | uej eadh 

T, ' iern is ho * ah "*«• ditiionav-y 
that assotiates the time with its ' 
toluiMh headiba. < 

\ 





All you need to do is work out how to populate the inner dictionary with the 
row data and the associated columns headings.. .and you’11 have all the data 
you need. 

The trick in creating the data structure is to realize that each row, including 
the column headings , are of a fixed size: 50 items. Knowing this, it’s not much 
work to create the dictionary you need: 


Ko eha^es 
ave bceded 
heve. 


Cveate ahothev, 

empty die-fciobavy. 


Mth eadh itevatio*, 
V is -the duVVCbt 

dolumb bumhcv. 


row_data = {} 

with open (’PaceData.csv 1 ) as paces: 

column_headings = paces.readline() .strip () .split (’, ’) 
column_headings.pop(0) 
for each_line in paces: 

row = each_line.strip().split( 
row_label = row.pop(O) 
inner_dict = {} 

for (i)in range(len(column_headings)) : 

\lnner_dict [row[i]] = column_headings[i] 
row_data [row_label] = inner_dict 

A 


I ! 


) 



t-ets ho-fc havd- 
t°At $ 0 , ta\t the 
value ihs-fcead. 



} 


f \ssodiate the 
tolumb headib^ 
with the time 
value -fvom the 


vow- 
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idle session 


— d /An IDLE Session- 

Go ahead and add the extra dictionary populating code to your program. Let's remove ali of those print () 
statements from the end of your program, because you'll use the IDLE shell to test your code. Run the code by 
pressing F5 or by selecting the Run Module option from the Run menu. Use the dir () BIF to confirm that your 
program code executed and that a collection of variables have been created in Python's namespace: 


»> dir () 

['_builtins_', ' _doc_', 

'inner dict’, 'paces', 'row' 


name , 

'row data' 


_package_' 

'row label'] 


'column headings', 'each line', 'i', 



AII of your todes variables exist 


W2 

34.3 

52.9 

—a 

51,1^ 

70.3 

77.5 

2mi 

5:00 

5:10 

e:zi ^ 

5:33 

5:44 

5k 

12.40 

13:05 

13:24' \ 

13:42 

14:66 

Stiti i 

iiTlo 


22:17 ) 

22:47 

23:16 

Ijn^ 

26.54 

27:30^ 



23:45 

20:24 

(isrs 

41:31 

42.27 

f 

r 43:24\ _ 

44:23 

45:23 


44.45 

45.45 



47:51 

43:56 

f ZS1H ) __. 

56.20 

57.45 

(59:M^ 

1:00:23 

1:01:45 1 

iTTrai! \\ 

59.40 

1:01:69 


^TOZT^2 

1:03:56 

1:05:23 

25K 

4^11:43 

1:13:2IL 

1:14:59 

1:15:40 

1:16:24 | 

Mfc 

1 :27: " 

T7i0:66 

1:31 'M 

1:33:11 

1:35:17 

Marathon 

2:05:34 

2.05:24 

2:11i17 

2:14:15 

2:17:16 








Take another look at (part of) the spreadsheet data file above, and let's try and find the column heading 
associated with the 43:24 time on the 15k row. Let's then use the column heading to find the predicted time for a 
20k run: 


»> column__heading = row_data [' 15k' ] ['43:24'] 

>» c i T -h.^„ g wUb Wadmj , s £omttl7 as "eu" 

O 1.1 


»> prediction = [k for k in row_data['20k'].keys() if row_data['20k'][k] == column_heading] 
»> prediction 

[' 59 : 03 '] 


A time o-f '5^:03" is torrettly predicted, too. 
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dealing with complexity 


Pissect thc predictiow code 

Let 5 s take a moment to review what just happened at the bottom of the IDLE 
Session from the last page. This line of code is a double-dictionary lookup on the 
dictionary-of-dictionaries stored in row_data: 

This is a ditWavy kcy 
vji*bhm Vo^jdata 




column heading = row data['15k’]['43:24’] 


Afid "this is dho^hcr 
di^tiohav-y key, 
indexes ih-fco 
thc dic-fciohary ^ 

Vow_datar/^kT. 


? 

value assodiated «rth the t^t v 
is the* asstyed to tt eolw»«_headm$ • 


Working out the predicted time in the 20k row of data involves finding the 
key in the row 5 s dictionary whose value is set to the just-discovered value stored 
in column heading. 



*r. a r e 'd eresied «dy ih data that 

satis+ics this ^ohditiohal. 


L 


prediction = [k for k in row_data['20k'].keys() if row_data['20k'][k] == column_heading] 

V 


t , , 

This is the data 701 * ve seavthm<y 


A conditional list comprehension is put to good use here. Recall that the list 
comprehension syntax is a shorthand notation for a for loop. The loop searches 
through the data in the list of keys associated with the dictionary stored at 
row_data [ ' 2 Ok ’ ]. If the value associated with the key (in k) is the same 
as column_heading, the value of k is added to the comprehensions results, 
which are then assigned to a new list call predicton. 

There’s really an awful lot going on in that comprehension. 
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409 
















list comprehension 



Sweet mother of all things 
Python! Whafs going on 
here? I think my brain is 
going to explode... 


Don’t let the list comprehension put you off. 

Recall that you can always rewrite a list comprehension using 
an equivalent for loop... 

Ummm.. .now there’s an idea. 
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dealing with complexity 



Rewrite each of the list comprehensions on this page to use a for 
loop. 


times = [t for t in row data['Marathon’].keys()] 


headings = [h for h in sorted(row data['lOmi'].values(), reverse=True)] 


time = [t for t in row data['20k'].keys() if row data['20k’][t] == '79.3'] 
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for loop 



You were to rewrite each of the list comprehensions to use a for 
loop. 


times = [t for t in row data['Marathon 1 ].keys()] 


Start With a* -> times =1 □ _ Tuv-h the 

empty list .. . dietiohav-y^s keys 

-f oY ea£h__t *m v-ow_datafA1av-athoK J.keysO: i*to a list 


times.apper\d(eadh_t) 

With eadh iteratio», appe»d the key (whith is a time 
value) o»to the times list 

headings = [h for h in sorted(row data['lOmi'].values(), reverse=True)] 


s-tart *ith a» --- J 7 headm 3 s - U Twr,n the dic ^ ha ' r y' s «lues i*to a list... 

em ? t7 list- 

(w eadh h in son-tedfrow datariOm/J.valuesO, vevevse—7Vue) ; 


■ ' .....•••• S1< « to's<»-t'the values m 

h d, ^ S Ppe .^ . .««i crder to^et &*l>- 


Y 


iVith eaeh iteratio», appe»d the value (whieh is a 
£olum» headi»g) o»to the w times w list 


time = [t for t in row data[’20k'].keys() if row data['20k'][t] == '79.3'] 


stav-t *<th a» -—=7 ' tiwe - ° C Tun the di ^ ioir,a ' r y' s k eys <»b> a list. 

emfty ^ov eath_t in vovj _dataTZOkYkeysO: 

i-f row_datarZ0k'JCeadh_tJ 
Theire^s a timeappe»d(eadh_t) 

de-fihite pattevn ..X'. , 

erwevainfl hev-e- © l Mth eatVi itevation, fchedk to sce i4r the 

toUn headin^ (the value part *C the 
didtionav-y) e«\uals "713" and it it does, 
append the ti*«e to the list- 
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dealing with complexity 


ftet input frow your user 


Now that you have your data within a Python data structure, it’s time to ask 
your user what it is they are looking for. 

Specifically, you need to know three things: the distance run, the time recorded, 
and the distance a prediction is required for. 

When you get to move your app onto Android, you can use a nice graphical 
dialog to ask your user for input, but for now, let’s quickly create a text- 
based user interface, which will allow you to develop and test the rest of the 
functionality required from your application. When you’re done, youTl create 
the Android app. 


Use mputO for input 

Python has the input () BIF that can help here, which is used to display 
a prompt on screen, and then accept keyboard input, returning what was 
entered as a string to your code. 




u4 


/ 


f An (DLE Session 


Using the input () BIF is best demonstrated with some examples: 

»> res = input('What is your favorite programming language 

What is your favorite programming language: Python 

>x> res _ The entered data is asstyed to Vs" 

'Python' 



a«d it's a STRI f^- 



hovide the prompt to display 
to you\r user. 


The input () BIF returns a string, which has been stripped of any trailing newline character, which would 
typically be included at the end of any input string. It is important to note that any input is returned as a string, 
regardless of what type of data you think you might be entering: 


The entered data is assigned to w age” and 
it s a string, even though you might want to 
treat it like it s a number. 



Convert the input to the type you ^ 
need dtfORt usin 9 the data- 


»> age = input ('What is your age: ') 


What is your age: 
»> age 
' 21 ' 

»> in t (age) 


Ctdito^s note : Vcah... 
dv-edm on> Paul- ©-J 


21 
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input error 


(retting input raises an issue... 

It’s not hard to use input () to get the, um, er.. .input you need. Here’s your 
code from earlier with three calls to input () added to interact with your 
user. 


maratho n. py- /Users/barryp/Head F i rstPython/ ch apter 11 /marathon.py 

row data = { } 

with open( 1 PaceData»csv 1 ) as paces: 


column headings - paces.readlinef),strip().split( 1 , 1 ) 
column headings-pop(0) 


for each_line in paces: 

row = each line.strip().split( 1 , 1 ) 
row_label = row.pop(0) 
inner_dict = 

for i in range(len( column headings)): 

inner dictjrowji]] = column headings[i] 
row_ data [ row__label ] = inner dict 


TKeres U tW.s, as 

wsev-mtevaaio»» wth mfwtO 
does^-t yt "«*<* easiev 
•fchis.. 


distance^run = input('Enter the distance attempted: 1 ) 
recorded time = input ( r Enter the recorded time: 1 ) 

predicted distance = input ('Enter the distance you want a prediction for: 1 ) 


Ln: 20 Coi: 0 


A 


When your program runs, your user enters some data, and look what happens\ 




Python Shell 


»> ================================ RESTART 

»> 

Enter the distance attempted: 20k 
Enter the recorded time: 59:59 
Enter the distance you want a prediction for 
»> 

»> 

»> row^ data \ distance _run ] [ recorded time ] 
Traceback (most recUTTt chtt Fctst; : 

File ,, <pyshell#76> n f line 1, in <module> 


Marathon 



row data[distance run][recorded time] 
KeyError: '59:59' 

T 



A w &y£v-v-ov-” exdepiion has 

beer» v-aisedL.bu-t v/hy? 


Ln: 205 Coi: 4 
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dealing with complexity 


If ifs not in the dictionary, it canT be found. 


The data in the row data dictionary originally comes from the spreadsheet 
and is read into your program from the GSV file. 

If the data value entered into the recorded_time variable is in the dictionary, 
things are going to be fine, because there 5 s a match. However, if the data entered 
into the recorded_time variable doesn’t match anything in the dictionary, 
you’11 get a KeyError. 


But how is this “problem” handled during training? 



M 




The enieved ii*e 4v a ZOk 

talis betweeh these Uo 
values oh -the pace sheei 


1UX 

26.64 

27«30 

Z6:93 / \ 

25:45 

23.24 

ish 

411:31 

■iziir 

43:24 ^ \i 

44:23 

45:23 

lOfllE 

4445 

4545 ^ 


<7:51 \ 

45:55 

2flfc 

5G:Z9 

57.45 V 

59:93 ) ( 

1:99:23 ) 

1:9145 | 

Ii.Imi 

5940 

1:Dl:ft& 

1T5ZT32 

iTL r 4.a& 

1:95:23 


1211:43 

1:13:20 

1:14:50 

1:15:40 

1:15:2-1 | 

Mtti 

1:Z7:m 


1:31:55 

1:33:11 

1:35:17 

Marath&n 

2:05:34 

2.03:24 

Z:11j17 

2:14:15 

Z: 17:16 
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c/ose enough 


Search for the closest match 



All you need to do is search the row of data for the closest match, right? 
And guess what? The Head First Gode Review Team think they have some 
functions that might help here. 


Theres nothing better 
than sharing our code 
wi+h our fellow Python 
programmers. Check out 
our "f ind it" module. 


TWis dode 'is 'm a tile talled 

"tmdj-tn” air,<1 Vf* ta * 

dovmload a do\>7 W» iliis 
book's suppovt 'websiie. 







findJt.py - /Users/barryp/HeadFirstPython/chapterll/findjt,py 


def findclosest (look for, target data): 


/f^ def whats the di 
l y~if first == 

return(G 


di££erence( first, second): 
== second: 



G) 

elif first > second: 

return( first - second) 
else: 

returnf second - first) 


Were s a* exa«,p| e o-P a ncshcd 
iuhdiion, whidh is allov/ed 

' h fac* i\wo values, 

rr ' b,h ^ ,on k ‘ e i l| i r ns tbe 
di+ievchCe betwee* them. 


max diff = 9999999 
for each^thing in target data: 

diff = whats the difference(each thing f look for) 
if diff == 0: 

found it = each thing 
break 

elif diff < max diff: 
maxdiff = diff 
found it = each thing 
return( £ound_it) 

bu“t i*t >woyks. 


Tbis may noi be tbe »>°st 
ettidieni seavdb dode evev 


Ln: 22 Cof' 0 


M 


Tbe "Pmd_dlosest” 

•fwndiion does a siwfle 
Imeav seavdb, vetuvmir^ ^ 
ibe value m "tav$et__data 
ibat r«ost dlosely »>atdbes 
ibe tt look_W’ av^wmeni- 
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dealing with complexity 




WAkN 


What do you think has gone wrong here? Why 
does the find closest () function crash when 
asked to work with data from your CSV file? 
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time trials 


The trouble is with time 

The data in your GSV file is a representation of timing values. Rather than 
actual numbers, the values in the GSV are strings. This is great for you, 
because you understand what the representation means. Python, on the other 
hand, sees the data only as strings. 

When you send your data to the f ind_closest () function, Python 
attempts to treatyour strings as numbers and chaos ensues. What might work 
would be to convert the time-strings into numbers. But how? 



When I have to work with 
times, I always convert my 
time strings to seconds 
first... 


Yeah, of course! 

Didn't we write the 
"tm2secs2tm" module to 
handle this sort of thing? 
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dealing with complexity 


The time-to-secowds-to-time module 


The Head First Gode Review TeanTs generosity knows no bounds. Sure enough, their 

rather strangely name tm2 secs2 tm. py module looks like it might help. ^jvab a £opy o-f -this todt «fi 

tferes ibe ^' s su PP°*"t websiie. 


vor* 




tm2secs2tm.py - /Users/barryp/HeadFirstPython/chapterll/tm2sec52tm.py 


import time 

def £ormat_time( time string): 
tlen = len(time string) 
if tlen < 3: 

original_format = ' %S 1 
elif tlen < 6: 

original_format = '%M:%S 
else : 


Tbis WW ensuves ibat ali W are W-atted m 
"Hrt:/V|/l4:SS" Wat Tbis belps keep ti»my sfefle wbe* do.^ 

tohvev-siohs io se£ohds. 


original_format = '%H:%M:%S r 

time string = time.strftime( 1 %H:%M:%S 1 , time.strptime(time string, original_format)) 
return( time string) 

a "tirne stri^j", tower i ii io a value m seto»ds. 



def time2secs (time string): 

time string = format = time(time string) 

(hours, mins, secs) = time string.split( 1 : 1 ) 

seconds = int(secs) + ( int (mins)*60) + (int (hours)*6Q*60) 

return( seconds) , , W i. 

^ __ . CohVev~t a value m sedohds *to a sformj • 

def secs2time( seconds): ^ 

return (time.strftime(' %H:%M:%S' , time.gmtime(seconds))) 


Ln: 24 Coi: 0 





Now that you have the tm2secs2tm.py and find_it .py modules, let’s create a function 
that uses the facilities provided by these modules to solve your searching problem. Your new 
function, called f ind_nearest_time (), takes two arguments: the time to look for and a 
list of times to search. The function returns the closest time found as a string: 


The £ode you 
need has beeh 
started -for 
you. 



from find_it import find_closest 
from tm2secs2tm import time2secs 

def find nearest time(look for. 


, secs2time 


target data): 


Whlike ih -the previous 
Captar, i-fc is possible 
i° do wha-fc you heed " 
io do hev*c ih ohly 
-fouir libes o-f £ode. 
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time to string 



ExwctSe 

§ 0 (.|#tl 0 H 


Now that you have the tm2secs2tm.py and f ind_it .py modules, you were to create a 
function that uses the facilities provided by these modules to solve your searching problem. Your 
new function, called f ind_nearest_time (), takes two arguments: the time to look for and 
a list of times to search. The function returns the closest time found as a string: 


i^ovt the 
team s dode 




from find it import find closest 


from tm2secs2tm import time2secs, secs2time 


The -fuhC-tion -takes two 
avgumehts, a time stvmg 
a list o-f time stvihas 

def find_nearest_time(look_for, target_data): J 

Convevt the time stvmg ~ ~ „ ' r , 

you ave lookmg -fov mto its ' wha ' t f’ ,rne . 2 - sedsC|ook - + °' r; c 

e<\uivale«t value m se£o«ds. , r ,. - . u) C i i iJiit -Convevt the Imes ot time 

wheve — ttir*ez.seds(t/ tov t m tavg>et_dataJ N stvmy 'm*to sedonds* 

Call w -f md_tlosestO”, ' ves =■ -f'md__dlosest(what, v/heve) 0 , 

the doywevted data.. . * n c ' oses ^ ^^ to the dallmg 


vetuv*(seds2.time(ves)) 


dode, a-Ptev doKvevti*<) it badk to a t 
stvih<y 


Ime 


r- « 




/ 


An (DLE Session 


Let's try out your code at the IDLE shell to see if your time "problems" have been resolved: 


^ctre s some o-f youv* 

pade data. Let s 
wovk with data tvom 


15X 

26..S4 

Z7.3D 

23: M 

23:45 \ 

Z9:Z4 

15fc 

41:31 

izizr 

43:24 

44:23 \ 

45:23 


44:45 

45.45 

45:45 

iT.it ) 

43:55 

1 MU ' r 

56:20 

57.45 

59: D3 


1:01:45 | 

l3.iSh 

53.40 

1:Dl:H 

1.02:32 

1:03:55 

1:05:23 

isk 

imi‘43 

1:13:25 

1:14:5* 

1:15:40 

1:15:24 | 

33H 

1:27:15 

1:10:55 

1:31:55 

1:33:11 

1:35:17 

Marath&n 

2:DS:14 

2.05:24 

Z:11j17 

2:14:15 

2:17:16 








»> f ind_nearest_time ('59:59' , 
'01:00:23' 

»> find_nearest_time (' 1: 01: 01' 
'01:00:23' 

»> find_nearest_time (' 1: 02 : 01' 
'01:01:45' 

»> find_nearest_time ('57:06' , 
'00:56:29' 


['56:29', 

, ['56:29' 

, ['56:29' 

['56:29', 


'57:45', '59:03', '1:00:23', '1:01:45']) 

, '57:45', '59:03', '1:00:23', '1:01:45']) 

, '57:45', '59:03', '1:00:23', '1:01:45']) 


6jveat! 
This 
appeavs 
to be 


'57:45', '59:03', '1:00:23', 


'1:01:45' 



wovk 
«f mC* 
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dealing with complexity 



With all this code available to you, it's an easy exercise to put it all together in your program and 
produce a complete solution to the Marathon Club's prediction problem. Let's take it for a test run. 


^ ^ 


marathon.py - /Users/barryp/HeadFirstPython/ehapterl 1/marathon.py 



from find it import findclosest 
from tm2secs2tm import time2secs, secs2time 

def find nearest time( look_for f target data): 
what = time2secs (look_f or) 

where = [ time2secs ( t) for t in target_data] 
res = find closest(what, where) 
return( secs2time(res)) 

rowdata = { } 

with open( F PaceData.csv r ) as paces: 

column headings = paces-readline().strip().split( 
column headings.pop(0) 

for eachline in paces: 

row = each_line.strip().split( 1 , 1 ) 
row_labeI - row.pop(Q) 
inner_dict = {} 

for i in range(len( column_headings)): 

inner dict [ row[i]] = column headings [i] 
row_data[row_label ] = inner^dict 

distancerun = input ( 'Enter the distance attempted: 1 } 
recorded_time = input( 'Enter the recorded time: ' ) 
predicteddistance = input ( ' Enter the distance you want a 


This to&t is uscd w as is" 


pmd ite 

WrtWm “the data- 

-the 

£olumh headih . 

r ediction for: F ) 



closest_time = find nearest_time(recorded_time, row data[distance run]) 
closest_column heading = row data [ distance run][closesttime] 


prediction = [k for k in row data[predicted distance].keys() 

if row data [ predicted distance] [k] == closest_column_heading] 

print( F The predicited time running F t predicted distance t F is: F + prediction [ 0 ] + F .') 



■Scair^h -Polr 

3 predicted 
at 

the desiv-ed 
distale ahd 
display i-fc oh 
s£v-eeh. 


Ln: 37 Coi: 0 


A\ 


* ^ ^ 


Python Shell 


RESTART 


»> =====^=====^==========^======== 

»> 

Enter the distance attempted: 2Qk 
Enter the recorded time: 59:59 
Enter the distance you want a prediction for 
Traceback (most recent call last): 

File 11 /Users/barryp/HeadFirstPython/chapterll/marathon.py" , line 31 
cio"sest cCrras^heading = row data[distance_run][closest time] 

I KeyError: ' 01: 0G:23>^ ^__ 


Marathon 


Tv-y out youv fv-o^va»» wth the same 

data 'm^ut -fvomn cav-liev-. 


in <module> 


A^othev" u ^eyEv\rov- ,; ... 


Ln: 13 Coi: 4 


After all that, you’re getting the same error as before. Bummer. 


you are here ► 
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more time trials 


The trouble is stjjl with time... 

Or, to be more precise, with how the tm2secs2tm. py module formats 
time strings. Take another look at the results from the previous IDLE Session. 
Do you notice anything strange about the results returned by the call to the 
find nearest time() function? 



When your code takes one of these returned values and tries to index into 
your dictionary, there 5 s no mateh found, because your dictionary 5 s keys do not 
confirm to the HH : MM : SS format. The solution to this problem is to ensure 
that every time you use a time-string in your code, make sure it’s in HH : MM: SS 
format: 

marathon.py - /Users/barryp/HeadFirstPython/chapterll/marathon.py 


from find_it import findclosest 
from tm2secs2tm import time2secs, secs2time. 


def find nearest time( lookfor, targetdata): 
what = time2secs(lookfor) 
where = [time2secs(t) for t in target data] 
res = findclosest(what, where) 
return (secs2time(res)) 

rowdata = {> 

with open( ' PaceData.csv ' ) as paces: 

columnheadings = paces.readline().strip().split( 
columnheadings.pop(0) 

for eachline in paces: 

row = each_line.strip().split) 
row label = row.pop(O) 
inner_dict = {} 

for i in rangeJJ,^rrfCoiuiim2 fteaG!rr S ; ^D 


inner dic Z[ format_time(row[ii/] 
row data[ ^ ] - --_ 

distancerun = input('Enter the distance attempted: 
recordedtime = input('Enter the recorded time: 
predicted_distance = input ( ' Enter th*> Histance^du want 




the "•Covmat_t ,rne ^ 

•fw»ttio« vro*n the 
"tmZsedsItio f/ ">o<Ue- 

Wse the tuhCtion to ensuve the times 
used ihtevnally by your Code are 
+ovmatted rn "WtM/VtSS" &rr»at. 

gs [ i ] 


prediction for: ’ ) 


closesttime = find_nearest_timt\ s format_time(recordedtime), rowdata(distancerun]) 
Closest COlumn heading = - ] [ r-1 nspsf r i mp ] 

prediction = [k for k in row_data(predicted distance].keys() 

if row_data[predicted_distance][k] == closestcolumnheading] 

print(’The predicited time running ’ + predicteddistance + ' is: ' + prediction(0] + ’.') 


Ln: 37 Coi: 0 


A 
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dealing with complexity 



Let's try your code one more time. Hopefully, now that all of the time strings within the system 
conform to HH: MM: SS format, your code will behave itself. 




Python Shell 


Python 3.1.2 (r3l2:7936QM, Mar 24 2010, 01:33:18) 

[ GCG 4.0.1 (Apple Inc. build 5493)] on darwin 

Type 11 Copyright " , "credits" or "license( ) " for more Information. 

»> =»=»=»=«»=»»»==»»»=» RESTART =-============-=-=============== 

Enter the distance attempted; 20k _. ~^ IS ,s pv-cvious -fces-fc, whi£h 

Enter the recorded time; 59:59 

Enter the distance you want a prediction for: Marathon / c / • 

Traceback (most recent call last) : 

File " /Users/barryp/HeadFirstPython/chapterll/marathon.py " , line 31, in <module> 
closest column heading = row data[distance run][closest_time] 

KeyError: '01:00:23' 

>» ================================ RESTART ================================ 

Enter the distance attempted: 2Gk ^--* TWis dv-ound) youv ^ekaves 

Enter the recorded time: 59:59 wovks -pmC- 

Enter the distance you want a prediction for: Marathon rtselF ano 
The predicited time running Marathon is: 02:14:15. 

»> ================================ RESTART ================================ 

»> 

Enter the distance attempted; 5mi Av\o4-U^ . C- 

Enter the recorded times 23:45 nwzncr test doh+i 

Enter the distance you want a prediction for: lOmi^ woirki 

The predicited time running lOmi is: 00:50:02. 

»> ================================ RESTART ============================= 

»> 

Enter the distance attempted: lOk 

Enter the recorded time: 32:15 

Enter the distance you want a prediction for: 25k * , P- n +«4: w\dkes sure 

The predicited time running 25k is: 01:25:49. flnd 

»> 


: 'hg wcll. 


-that -thihgs ave 



Ln: 31 Coi: 4 

__ I /V* 


This is working well. You Ve solved your applicationi Central problem: your 
program reads in the spreadsheet data from the GSV file, turns it into a 
dictionary of dictionaries, and lets you interact with your user to acquire 
the recorded time at a particular distance before predicting a time for 
another distance. 

Not counting the code provided by the Head First Code Review Team, 
you Ve written fewer than 40 lines of code to solve this problem. That’s 
quite an achievement. All that 5 s left to do is to port your program to the club 5 s 
Androidi phones. 

And porting to Android won’t take too long, will it? 


you are here ► 
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android interface 


Fort to Android 


Your code is working great. Now it’s time to port your text-based Python 
program to Android. Most of your code doesn’t need to change, only the 
parts that interact with your user. 


Obviously, you’11 want to make things as easy to use as possible for users of 
your latest Android app, providing an interface not unlike this one. 



Cancel 


aio 


Marathon 


Select 


Use HH:MM:SS format: 


Pick a distance to predict 


I. S-tav-t 

a distante 




Thcsc ave bo-fch 
dialog boxes. 


'i. Sclcd-t a 

distale -to 
pvedifrt *to- 


2- fcnier Ihe 
'retorded lime. 


Vialo^W 

dialog box. 


This is 


This is a ^ 
"dialo^c-tl^uV 
dialo^ bo%. 


^ Aflev Ihe 
lookup, display 

the pv-cdi^-tcd 

■fcir*e. 
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dealing with complexity 


Your Android app is a buwch of dialogs 


Your Android app interacts with your users through a series of dialogs. Other 
than the single dialog that requests data from your user, the other three share 
certain similarities. You can take advantage of these shared features by 
creating a utility function which abstracts the dialog creation details: 



The dialog s 
trtle 


The list o£ 
values -to dlsplay. 


The dialog 
Creatio* 
me^hod hame 


The texi *C oY 

"the butfcohs, 

wi-th de-faults. 


def do_dialog(title, data, fune, ok=’Select’, notok='Quit’) 
app.dialoqCreateAlert (tit le 
fune(data) 

app.dialogSetPositiveButtonText(ok) 
if notok: 

app.dialogSetNegativeButtonText(notok) 
app.dialogShow() 

return(app.dialogGetResponse().resuit) 


Pisplay the dialog a«d theh 
the seleeted item. 



Assume the existence of a list called distances, which contains 
the row distance labeis ( 2mi, 5k, 5mi, and so on). In the space below, 
provide the two calls to the do_dialog () function needed to create 

the two dialogSetSingleChoicel tems shown on the left of 
the previous page. 


you are here ► 
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adding dialog 



You were to assume the existence of a list called distances, which 
contains the row distance labeis. In the space below, you were to 
provide the two calls to the do_dialog () function needed to create 

the two dialogSetSingleChoiceltems. 


do__dialog(”Pidk a distande”, distales, 


Provide -the list o( 
itcrhS to display. 


pv-ovide ‘the ... . . .’’. 

dialog title.- - 

do__dialog( ;; pidk a distande to predidt”, dista^des, -Provide the 

Dilio: do ii ali " >) . -type of dialog 

agai» (o\r -the -^ appdialog£etS‘mgleChoide|tems) -to use. 

° d alog. do dialog (’ The predicited time running ' + predicted_distance + 

' is: ', prediction, app. dialogSetltems, "OK" , None) 


Heres 
ano-ther 
e%ar*ple 



^f IWe taW, 

you have to build up the dialog title 

, So g va ^bles (that you'11 ,eed to 
bave ev-eated -Piv-st). 


A- 


Wse a di-fterent dialog 
tveating n>etbod "tbis 
-time- 


f 


OvevHde the 
de-Pault values ■{•ov* 
the dialog s buttohs. 


(ret your Android app code ready 

To use your dialog creating code, import the necessary libraries, define 
some constants, create an Android object, and reuse some code from 
earlier in this book: 


import time ^' Po youV iwports. 

import android 


distances = [ '2mi', '5k', '5mi', '10k', '15k', '10mi', '20k', **- 




'13.Imi', '25k', '30k', 'Marathon' ] 

hello msg = "Welcome to the Marathon Club's App" 
quit msg = "Quitting the Marathon Club's App." 


Oeate a list ot 
v-ow labeis. 


4l 


app = android. Android () 


Cireaie a» Axdvoid 
app objeet- 


De-Pihe two 
I -Priehdly messages. 


def status_update(msg, how_long=2): 

app .makeToast (msg) i*r 1S take* "as-is” 

•fvom eav-liev- ’m tbis book- 


time.sleep(how long) 
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dealing with complexity 



j\ndr®id Poo] puzzjc 

Your job is to take the code from the pool and place 
it into the blank lines in your Android app code. 
You can assume that the row_data dictionary 
exists and has been populated.The variables 
shown at the bottom of the last page have also 
been created, and the status_update () 
and do_dialog () functions are available 
to you. Your goal is to arrange the code so that it 
implements the Ul interactions you need. 


status_update(hello_msg) 

resp = do dialog( M Pick a distance", distances, 



distance run = 

distance run = distances[distance run] 

= app.dialogGetInput("Enter a " + distance run + " time:", 

"Use HH:MM:SS format:").resuit 

closest time = find nearest time(format time( ), row data[distance run]) 

closest_column heading = row data[distance run][closest_time] 
resp = do dialog("Pick a distance to predict", distances. 


The dialog^e-t/nputO *, e -thod 
displays -the inpu-fc dialog box. 


predicted distance = 

predicted distance = distances[predicted distance] 
prediction = [k for k in row data[predicted distance].keys() 

if row data[predicted distance][k] == closest_column heading] 
do dialog('The predicted time running ' + predicted distance + ' is: ' , 


prediction, app.dialogSetItems, "OK", None) 


status update(quit msg) 
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out of the pool 



P®®] puzzjc ^^luflan 

Your job was to take the code from the pool and 
place it into the blank lines in your Android app 
code.You were to assume that the row_data 
dictionary exists and has been populated.The 
variables you need also have been created, and 
the status_update () and do_dialog() 
functions were available to you. Your goal was 
to arrange the code so that it implements the Ul 
interactions you need. 


M yowr usev io pick a 
distale ih e | is t 0 ( 
labeis. 



status_update(hello_msg) 

resp = do dialog( M Pick a distance", distances, app.dialogSetSingleChoiceltems) 
if resp['which'] in ('positive'): ^ ss -,<y* -the selefrted distante tabel 

distance_run = app.dialogGetSelectedltems().resuit[0] io "dista*tejru* • 

distance run = distances[distance run] ^ Sr^ Ask youV usev enier ihe 

recorded time = app. dialogGetlnput ("Enter a " + distance run + " time:", ^Otdid iime. 

"Use HH:MM:SS format:").resuit 

closest_time = find nearest time(format time(recorded time), row data[distance run]) 
closest column heading = row data[distance run][closest time] 

resp = do dialog("Pick a distance to predict", distances, app.dialogSetSingleChoiceltems) 
if resp['which'] in ('positive'): A i 

. \ y oulr «e\r io pidk a 

predicted_distance = app. dialogGetSelectedltems (). resuit [ 0 ] v —distah£e -(Vo I st 

predicted_distance = distances [predicted_distance] labeis -fco pvedi^t "fco 

Look up ihc prediction = [k for k in row_data[predicted_distance].keys() 

plredu^fcioh. ^ if row data[predicted distance][k] == closest column heading] 

do dialog('The predicted time running ' + predicted distance + ' is: ', 

prediction, app.dialogSetltems, "OK", None) pispldy tbe pvcdifrtcd t***C at tbc 


^-3 

Wovk out 
what toWnn 
beadin$ to 

USC- 


status update(quit msg) 


seletied distale io your usev- 
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dealing with complexity 


Put your app together... 

You now have all the code you need to create your app: 




marathonapp,py - /Users/barryp/HeadFirstPython/chapterl 1 /marathonapp.py 


impo r t t ime 
impo r t and r o id 


Do your impovts. 


from find it import find cloeest 

from tm2Becs2tm import time2secs, secs2time, format time 


def findnearest time[look for, target data): 

what — timeZsecs [ look_for } / 

where = [time 2 secs(t } for t in target data J a 
ree ■= find closest ( what, where) 
return ( secs 2 time ( res )) 


yidludc your n-fmd^carcstO 
-functio*. 


distances = [ ' 2 mi ' , ' 5k " , 

1 13 ,Imi 1 , 1 2 5k 1 


, _i tl r ! 5mi‘, 1 1 0 k 1 , 1 1 5 k 1 , ! 10mi\ 

, ' SBk 1 , '3Gk', 'Marathon' J 

hello_msg — " 'rfelcome to the Marathon Club ' b App" 
quit msg — "Quitting the Marathon Club ‘ b App." 


2 Gk 1 


, 6 — Dedare your dons-tanis. 


row data ■= {} 



w i th ope n ij ' / sdc ard / b 14 a / Bcr ip t b /P ace D ata.csv' ) as p ace b : 
cclumn headings — paces. re ad line ij ). Btrip () sp lit [ ', ' ) 
coluum headings.pop { 0 ) 
for each line in paces: 


SDCARD' i ? e daia Ue ° h the 

±VLMD .s speii+i£ h*A+ 6 xd- 


row — each line.atrip (}. split [ 


} 



row label — row.pop(O) 
inner d ic t — {} 

for i in range [ len (coluum headings)): 

inner dict[format time (row[i ] )■ ] - coluum headinga[i) 
row data[row label J = inner dict 


£j*-ab and pv-eprotess youv 
CSV data 


app = and ro id.And ro id [) 


S- 


de f status updateij ms g , how lo n g - 2 ) : 
app. makeToa&t i{ msg ) 
t ime . s leep i| how lo n g ) 



Creaie your A*droid app object a»d 
■ndlude your belpev- 4w£i. 


:iokis. 


def do dialog [ title, data, fune, ok-' Seleet' , notok= ' Quit ' ) : 
app. d i a lo gC re a te Ale r t ij t i t le ) 
fune[data ) 

app. dialog£etPositiveButtonTe,Kt ( ok ) 
if notok: 

app.dialogSetHegativeButtonText [ notok ) 
app.dialo g £ how [) 

return [ app.dialogGetResponse [) .resuit) 


status update(hello m&g) 


pisylay your Wl your uscr a*d yrodcss 
rcsultmj mtcrafcfcion- \ 


" time: 1 ' 1 , \f 


resp = do dialo g [ " Pick a dista nce ' , dista nce s , app.dialog£et£ingleC hoiceItems ) 
if resp [ 1 which' ] in [‘positive ): 

distance run — app.dialogOetEelectedltems [), resuit [ 0 J 
distance run - distances [ distance run) 

recorded time = app.dialogQetlnput ( Bnter a + distance run + 

"Use HK:MM:££ format: ). resuit 

desest time = find nearest time [ format_time ( recorded time), row data[distance runj) 
closest column heading = row data [ distance r un ) [ closest_time ] 

re sp = do dialo g [" Pick a dista nce to p redic t r , dista nce s , app.dialog£et£ingleChoiceItems ) 
if resp [ 1 which ‘ J in ['positive'): 

p redic ted dista nce — app.dialo gQe t £elec tedItems [ ) .re s u11 f 0 J 
p redic ted dista nce - dista nce s [ p redic ted dista nce ) 
prediction — [k for k in row data [ predicted distance ]. keys [ ) 

if row d a t a [ p redic ted dista nce J [ k J -■= clo se s t co1umn he adi n g J 
do dialog( The predicited time running + predicted distance + is: , 

p red ic t io n , app . d i a lo g £e 11 tems , " OK ' ! , None ) 

status update(quit msg) 


Ln: 67 Coi; 0 


A 
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te st drive 



lt's time to test your Android app on the Android Emulator before loading a working applicatiori 
onto a "real" phone. Start your Android emulator and begin by transferring your code and the 
files it needs onto the emulator's SDCARD. Use the adb command in the tools folder to copy 

marathonapp .py, f ind_it .py, tm2sec2tm. py and PaceData . csv to the emulator, and 
then take your app for a spin. 


2opy youv todc 
Snd i-b suffort 
(iles b be 
emwlabv wth 
bese tommands- 


File Edit Window Help CopyToEmulator 


$ tools/adb push marathonapp.py /mnt/sdcard/sl4a/scripts 
43 KB/s (2525 bytes in 0.056s) 

$ tools/adb push find_it.py /mnt/sdcard/sl4a/scripts 
7 KB/s (555 bytes in 0.069s) 

$ tools/adb push tm2secs2tm.py /mnt/sdcard/sl4a/scripts 
12 KB/s (628 bytes in 0.050s) 

$ tools/adb push PaceData.csv /mnt/sdcard/sl4a/scripts 
59 KB/s (4250 bytes in 0.069s) 


And theve it • 
wdVbm<) -for you 
to test it 



5554:droid2.2 


Qfifl<3 2:39 


Scripts 


SL4A r2 


«3* bluetooth_chat.py 

coachapp-KEEPER.py 
coachapp.py 
< 3 * findjt.py 
«3» get2inputsapp.py 

*3» hello_world.py 
•3» marathonapp.py 
•3* notify_weather.py 
*3» say_chat.py 
*3» sayjime.py 


MENU 


ALT 


SYM 


Go on. You know you want to: tap that app! 
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dealing with complexity 


Your app'$ a wrap! 

All that 5 s left to do is transfer your working Android app to the Marathon 
Club 5 s phones.. .and that’s easy when you use AndFTP. When you show off 
your latest work, the club 5 s members can’t believe their eyes. 






And there's no stopping you! 

You Ve put your Python skills and techniques to great use here. 

Whether you’re building an app for the smallest handheld device or the 
biggest web server, your Python skills help you get the job done. 

Congratulations! 
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CHAPTER11 


python toolbox 



Your Pythow Toolbox 

You’ve got Chapter 11 under your 
belt and you’ve demonstrated a 


mastery of your Python toolbox. 


Congratulations and well done! 


Pythofi L-inJo 

9 A W.tional” Itet do^vebensjor, 
is o*e tbat «eludes a tva'.U$ * 

statemwt, alloWme you to tM ^ 
items ave added to tbe ne* l«t as tbe 

£omfv-ehev\sioir\ v-uns. 

* List towfvebensions ea* be vewvitten 
as an e^wivalent "W U?- 



BULLET POINTS 


■ The input () BIF lets you prompt and 
receive input from your users. 


■ If you find yourself using Python 2 and in 
need of the input () function, use the 
raw input () function instead. 


■ Build complex data structures by 
combining Python’s built-in lists, sets, 
and dictionaries. 


■ The time module, which is part of 
the Standard library, has a number of 
functions that make converting between 
time formats possible. 
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dealing with complexity 


lt'$ time to go... 



This is just the begiwwmg 

We’re sad to see you leave, but there’s nothing like taking what you’ve learned and putting it to use. You’re 
just beginning your Python journey and you’re in the driver’s seat. We’re dying to hear how things go, so drop us a 
line at the Head First Labs website, www.headfirstlabs.com, and let us know how Python is paying off for YOU! 
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appendix^ lefoVers 




The Top Ten Things 
(we didn’t cover) 




You’ve come a long way. 

But learning about Python is an activity that never stops. The more Python you code, 
the more you’ll need to learn new ways to do certain things. You’ll need to master new 
tools and new techniques, too. There’s just not enough room in this book to show you 
everything you might possibly need to know about Python. So, here’s our list of the top ten 
things we didn’t cover that you might want to learn more about next. 
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pro tools 


#1: Usiwg a "professional" IPE 

Throughout this book, youVe used Python’s IDLE, which is great to use 
when first learning about Python and, although it’s a little quirky, can handle 
most programming tasks. It even comes with a built-in debugger (check 
out the Debug menu), which is surprisingly well equipped. Ghances are, 
however, sooner or later, you’11 probably need a more full-featured integrated 
development environment. 

One such tool worth looking into is the WingWare Python IDE. This 
professional-level development tool is specifically geared toward the Python 
programmer, is written by and maintained by Python programmers, and is 
itself written in Python. WingWare Python IDE comes in various licencing 
flavor: it’s free if you 5 re a student or working on an open source project, but 



More general tools also exist. If you are running Linux, the KDevelop IDE 
integrates well with Python. 

And, of course,there are all those programmer editors which are often ali 
youdl ever need. Many Mac OS X programmers swear by the TextMate 
programmer 5 s editor. There’s more than a few Python programmers using 
emacs and vi (or its more common variant, vim). Your author is a huge fan 
of vim, but also spends large portions of his day using IDLE and the Python 
shell. 
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*Z: Copmg with scopmg 


Gonsider the following short program: 


A global vavtable talled "nan>e"- 


scope,py - /U^^rs/barryp/Head First Python/Top Ten Things/scope, py 


name = "Head First Python 

def what happens here( ): 

print (name) 
name = name + 11 is a great booki " 

print (name) 

what happens here( ) Call the -('unfcbon- 
print (name) 


A -fuh^tioh whidh 

attcmp-ts -to rcad 

-fv-oria dhd w\ri-fce “to 
the global vaviablc 
talled w har*e”. 



Ln: 12 CoL 0 


A 


See what "nan, e " is set to 
a+teir the Wtion vuns. 


If you try to run this program, Python complains with this error message: 
UnboundLocalError: local variable ‘name’ referenced before assignment. . .whatever that 
means! 

When it comes to scope, Python is quite happy to let you access and read 
the value of a global variable within a function, but you cannot change 
it. When Python sees the assignment, it looks for a local variable called 
name, doesn’t find it, and throws a hissy fit and an UnboundLocalError 
exception. To access and change a global variable, you must explicitly declare 
that’s your intention, as follows: 


^ ' scope.py - /Users/barryp/HeadFirstPython/Top Ten Things/scope.py 


name = "Head First Python" 

def what happens here( ): 

print ( name) 
global name 

name = name + " is a great bookl 111 

print (name) 

what happens here() 
print ( name) 


Ln: 13 Coi: 0 




So«,e pvogvan,n,evs ■fj^ 

«\uite ugly. Cthevs think it* 
what don.es to pass when you 
watdh AJonty Python vevuns 
while designing youv pv-og»-an,n,ina 
language. Ho «.attev what 
evevyone thinks: this is what 
we ve studk with/ © 
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testing, testing 


#3: testing 


Writing code is one thing, but testing it is quite another. The combination of 
the Python shell and IDLE is great for testing and experimenting with small 


snippets of code, but for anything substantial, a testing framework is a must. 

Python comes with two testing frameworks out of the box. 

The first is familiar to programmers coming from another modern language, 
because if s based on the popular xUnit testing framework. Python’s 
unit te st module (which is part of the Standard library) lets you create test 
code, test data, and a test suite for your modules. These exist in separate files 
from you code and allow you to exercise your code in various ways. If you 
already use a similar framework with your current language, rest assured that 
Python’s implementation is essentially the same. 

The other testing framework, called docte st, is also part of the Standard 
library. This framework allows you to take the output from a Python shell or 
IDLE session and use it as a test. All you need to do is copy the content from 
the shell and add it to your modules documentation strings. If you add code like 
this to the end of your modules, they’11 be ready for “doctesting”: 




/t you IT tode is impov-fced 
as a module, -this tode 


if name 


u 


main 


does UOT vum If you nm 


import doctest 
doctest.testmod() 


/°UT module -fv-om the 
dommand lihe, youv -fcesfcs 


ITUK. 


O 


If you then run your module at your operating Systems comand line, your 


For more on unittest and doctest, search the online Python 
documentation on the Web or via IDLE’s Help menu. 


tests run. If all you want to do is import your module’s code and not run your 
tests, the previous if statement supports doingjust that. 



* 
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leftovers 


#4: Advanced language features 


With a book like this, we knew we’d never get to cover the entire Python 
language unless we tripled the page count. 

And let 5 s face it, no one would thank us for that! 


There 5 s a lot more to Python, and as your confidence grows, you can take the 
time to check out these advanced language features: 

Anonymous functions: the lambda expression lets you create small, one- 
line, non-named functions that can be incredibly useful once you understand 
what’s going on. 

Generators: like iterators, generators let you process sequences of data. 
Unlike iterators, generators, through the use of the yield expression, let 
you minimize the amount of RAM your program consumes while providing 
iterator-like functionality on large datasets. 

Custom exceptions: create your own exception object based on those 
provided as Standard by Python. 


Function decorators: adjust the behavior of a preexisting function by 
hooking into its start-up and teardown mechanisms. 


Metaclasses: custom classes that themselves can create custom classes. 
These are really only for the truely brave, although you did use a metaclass 
when you created your Sightings form using the Django form validation 
framework in Ghapter 10. 

Most (but not all) of these language features are primarily of interest to the 
Python programmer building tools or language extensions for use by other 
Python programmers. 

You might never need to use some of these language features in your code, 
but they are all worth knowing about. Take the time to understand when and 
where to use them. 

See #10 of this appendix for a list of my favo rite Python books (other than 
this one), which are all great starting points for learning more about these 
language features. 
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regexes 


* 5 : Regular cxpressiows 


When it comes to working with textual data, Python is a bit of a natural. 

The built-in string type comes with so many methods that most of the 
Standard string operations such as finding and splitting are covered. However, 
what if you need to extract a specific part of a string or what if you need 
to search and replace within a string based on a specific specification? It is 
possible to use the built-in string methods to implement Solutions to these 
types of problems, but—more times than most people would probably like to 
admit to—using a regular expression works better. 

Gonsider this example, which requires you to extract the area code from the 
phone_number string and which uses the built-in string methods: 


' * ' area_code,py - /Users/barryp/HeadFirstPython/Top Ten Thinqs/areajcade.pY 


phone number = "Home: (555) 265-2901" 

. u/w 

start = phone .number .find( ’ ( ’ ) ^_n«d the openmg • 

Calculate where the area todc is 
,h the s-fo-ihg. 

area code = phone number [ start:end] the area £ode- 

print( r The area code is: ' + area code) 


start = start+1 
end = start+3 


} 


Ln: 12 Coi: 0 






Je£C FvicdPs regular 
expressio* w biblc > 

is weII worth a 

look i-f Y ou 
Icarn r*ore- L-ook up 
i\\t module m 
Pytho^s dots, -boo. 


This code works fine, but it breaks when presented with the following value for 
phone_number: ___ 

l^hy does this phone number 
«use the program to tail? Try 
't see what happens... 

When you use a regular expression, you can specify exactly what it is you 
are looking for and improve the robustness of your code: 


phone number = "Cell (mobile): (555)-918-8271" 


^ ' area tode rfi.py - /Users/barrYp/HeadFirstPython/TopTen Things/area code re.py 

import re 

phone_number = "Home: (555) 2 65-2^0JJh^ - 

results = re.search(' \( ( \d{3))\ ) 1 f phone number) 
area code = results.group( 1 ) 

print('The area code is: 1 + area code) 



This looks a litti e strange, 
but this regular expression 
is looking for an opening T 
tollowed by three digits 
and then a elosing 
This specif ication is much 
">«re likely to find the 
todt and wor/t break 
as ‘KWy as -the other 
versioh ot this prograr». 


Ln: 10 Coi: 0 
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#6: More ow web fraweworks 

When it comes to building web applications, GGI works, but it 5 s a little 
old-fashioned. As you saw in Ghapter 10, Google’s App Engine technology 
supports GGI, but also WSGI and a number of web framework technologies. 
If you aren’t deploying to the cloud and prefer to roll your own, you have 
plenty of choices. What follows is a representative sample. My advice: try a 
few on for size and see which one works best for you. 

Search for the following terms in your favorite search engine: Django, Zope, 
TurboGears, Web2py, and Pylons. 


django 


The u ol d tirir»ev-s w ..but 
Aorti Ici ma-tuvi-ty -fool 
you: ihese are ^kihg 
web -rirarhewov-ks. 






^ea-tuves. 
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data mappers and nosql 


* 7 : Object relational mappers and NoSQL 

Working with SQL-based databases in Python is well supported, with the 
inclusion of SQLite in the Standard library a huge boon. Of course, the 
assumption is you are familiar with SQL and happy to use SQL to work with 
your data. 

But what if you aren’t? What if you detest SQL? 

An object relational mapper (ORM) is a Software technology that lets 
you use an underlying SQL-based database without having to know anything 
about SQL. Rather than the procedural interface based on the Python 
database API, ORMs provide an object-oriented interface to your data, 
exposing it via method calls and attribute lookups as opposed to columns and 
rows. 

Many programmers find ORMs a much more natural mechanism for 
working with stored datasets and the Python community creates and supports 
a number of them. 

One of the most interesting is SQL Alchemy, which is popular and included 
in a number of the web framework technologies discussed in #6. Despite 
being hugely popular anyway, SQL Alchemy is also interesting because it 
supports both Python 2 and Python 3, which makes it a standout technology 
(for now). 

If you find yourself becoming increasingly frustrated by SQL, check out an 
ORM. Of course, you have already experienced a similar technology: Google 
App Engine’s datastore API is very similar in style to those APIs provided by 
the major Python ORMs. 


SOLAlchemg 


There'$ NoSQl, too. 

In addition to database technologies that let you avoid working with the 
underlying SQL-based database, a new breed of technologies have emerged 
that let you drop your SQL database in its entirety. Known collectively as 
NoSQL, these data tools provide an alternative non-SQL API to your 
data and do not use an SQL-based database management system at ali. As 
these technologies are relatively new, there 5 s been more activity around 
Python 2 than Python 3, but they are stili worth checking out. CouchDB 
and MongoDB are the two most closely associated with robust Python 
implementations. If you like working with your data in a Python dictionary 
and wished your database technology let you store your data in much the 
same way, then you need to take a look at NoSQL: it’s a perfect fit. 


CouchDB 

• 1 reiax 


, mongoDB 
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*$: Prograwwmg ftUls 

In this book, youVe created text-based interfaces, web-based interfaces and 
interfaces that ran on Android devices. But what if you want to create a 
desktop application that runs on your or your user’s desktop computer? Are 
you out of luck, or can Python help here, too? 

WelL. .as luck would have it, Python comes preinstalled with a GUI-building 
toolkit called tkinter (shorthand for Tk Interjace). It’s possible to create a usable 
and useful graphical user interface (GUI) with tkinter and deploy it on Mac 
OS X, Windows, and Linux. With the latest version of Tk, your developed app 
takes on the characteristics of the underlying operating system, so when you 
run on Windows, your app looks like a Windows desktop app, when it run on 
Linux, it looks like a Linux desktop app, and so on. 

You write your Python and tkinter code once : then run it anywhere and it just 
works. There are lots of great resources for learning to program with tkinter, 
with one of the best being the last few chapters of Head First Programming, but 
since plugging that book would be totally shameless, I won’t mention it again. 

Other GUI-building technologies do exist, with the PyGTK, PyKDE, 
wxPython, and PyQT toolkits coming up in conversation more than most. Be 
warned, however, that most of these toolkits target Python 2, although support 
for Python 3 is on its way. Search the Web for any of the project names to learn 
more. 


0 V>, look : 'rt's one ot the 
£jU|s treated in "Head r ivst 
Pvo 3 v-an>n>iv'$".. and ves, I sa.d 
| vjouldnt «>e»tion THAT 
book a<\ain, b**t isnt this qUI 

beau-tihl? © V_ 
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your bugs, my bugs, and threads 


#9: Stuff to avoid 


When it comes to stuff to avoid when using Python, there’s a very short list. A 
recent tweet on Twitter went something like this: 



Threads do indeed exist in Python but should be avoided where possible. 

This has nothing to do with the quality of Pythoffs threading library and 
everything to do with Pythoffs implementation, especially the implementation 
known as C Python (which is more than likely the one yoffre running 
now). Python is implemented using a technology known as the Global 
Interpreter Lock (GIL), which enforces a restriction that Python can only 
ever run on a single interpreter process, even in the presence of multiple 
processors. 

What all this means to you is that your beautifully designed and implemented 
program that uses threads will never run faster on multiple processors even 
if they exist, because it carit use them. Your threaded application will run 
serially and, in many cases, run considerably slower than if you had developed 
the same functionality without resorting to threads. 

Main message: don ? t use threads with Python until the GIL restriction is 
removed. ..if it ever is. 
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#10: Other books 

There are lots of great books that cover Python in general, as well as 
specifically within a particular problem domain. Here is a collection of my 
favori te Python books, which we have no hestitation in recommending to you. 


/eludes a yreai ta%t study 
mvokmg ihe pwrtihj of a <W| e * 
rythoh 2 module io Py-fch Oh 3. 


ntixptn'3 vucis w V W 


Dive Into 

Python 3 




^«1 


I ^fxjthon 

MarkPiIgnm 






D. i i a M. EJcaifc-', 


Python 

Essential Reference 


-Tl"' Edilkui 


IJeyeloper* Lhwary 



detmitWe a« 5 <* ^ m vt! 

P , ytVio*' : «t * ^ * 



/*P you ave a 
sysadmih, theh 
-this is the Py-thoh 

book -fov you. 



Nl rrrir 


for Unix 


.andUmtx 

AtimkmtuiUM 


OREU.Y 




Python 




some ^"?' es 

»**Sf * MU HS* 

<a*ed ^ ’ 
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Symbols and Numbers 

404 Error, from web server 242 

405 Error, from web server 378 
>>> (chevron, triple) IDLE prompt 4 

: (colon) 

in for loop 16 
in function definition 29 
in if statement 20 

, (comma) separating list items 7 

{ } (curly braces) 

creating dictionaries 180 
creating sets 166 

= (equal sign) 

assignment operator 7 
in function argument definition 63 

(...) (parentheses) 

enclosing function arguments 29 
enclosing immutable sets 91 

+ (plus sign) addition or concatenation operator 138 
# (pound sign) preceding onedine comments 38 
@property decorator 250, 253, 285 
? (question mark) parameter placeholder 321, 350 
“...” or 2..’ (quotes) enclosing each list item 7 

or (quotes, triple) enclosing comments 37 
; (semicolon) separating statements on one line 38 

[...] (square brackets) 

accessing dictionary items 180, 212 
accessing specific list items 9, 18 
enclosing all list items 7,18 

A 

“a” access mode 110 
access modes for files 110, 133 


addition operator (+) 138 
Alt-N keystroke, IDLE 5 
Alt-P keystroke, IDLE 5 
AndFTP app 288-289 
Android apps 

accepting input from 278-282, 295, 304-307 
converting from Python code 424—430 
creating 274-277, 281-282 
data for. See JSON data interchange format 
integrating with SQLite 342—348 
running on phone 288—289 
scripting layer for. See SL4A 
troubleshooting 277 

Android emulator 

installing and configuring 260—262 
running scripts on 264-265, 272-273, 283 

Android Market 288 
Android Virtual Device. See AVD 
anonymous functions 439 
appendO method, lists 10,14 
apps. See Android apps; webapps 
app.yaml file 356, 395 

arguments for functions 
adding 52, 66-68 
optional 63-64, 134 

arrays 

associative. See dictionaries 
similarity to lists 9-10, 17 

as keyword 119, 138 

assignment operator (=) 7 

associative arrays. See dictionaries 

attributes, class 190, 194, 212 

authorization, user 389—393 

AVD (Android Virtual Device) 261, 291 
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B 

“batteries included” 32 
BIFs. See built-in functions 
BigTable technology 354, 359 
blue text in IDLE 4 
books 

Dive Into Python 3 (GreateSpace) 445 
Head First HTML with GSS & XHTML (0’Reilly) 
374 

Head First Programming (0’Reilly) 443 
Head First SQL (0’Reilly) 313 
Learning Python (0’Reilly) 445 
Mastering Regular Expressions (0’Reilly) 440 
Programming in Python 3 (Addison-Wesley Profes- 
sional) 445 

Python Essential Reference (Addison-Wesley Profes- 
sional) 445 

Python for Unix and Linux System Administration 
(0’Reilly) 445 

braces. See curly braces 

brackets, regular. See parentheses 

brackets, square. See square brackets 

BSD, running GGI Scripts on 239 

build folder 42 

built-in functions (BIFs). See also specihc functions 
displayed as purple text in IDLE 4 
help on 21 

importing of, not needed 55 
namespace for 55 
number of, in Python 21 

_builtins_namespace 55,71 

c 

cascading style sheet (GSS) 374-375 
case sensitivity of identihers 17 
cgi-bin folder 234, 235 

GGI (Gommon Gateway Interface) Scripts 217, 235, 243, 
253. See also WSGI 

location of 234, 235 
running 239 

running from Android 264—265, 272—273, 283 


sending data to 300-303 
tracking module for 248—249 
troubleshooting 242, 247-250 
writing 236-238, 244-246 
writing for Android. See SL4A 

egi library 300 

cgitb module 248-249, 253 

chaining 

functions 146, 172 
methods 142, 172 

chevron, triple (>>>) IDLE prompt 4 

chmod command 239, 253 

classes 189-191 

attributes of 190,194,212 
benehts of 189 

converting data to dictionary 285-286 
dehning 190—193, 194, 195—196 
inherited 204-209, 212 
instances of 190, 191, 194, 195-196 
metaclasses 439 

methods of 190, 195-196, 198-200 
in modules 209, 212 

class keyword 191, 212 

close() method, database connection 315, 350 

closeQ method, hies 75,103 

code editors 35, 436. See also IDLE 

colon (:) 

in for loop 16 
in function dehnition 29 
in if statement 20 

comma (,) separating list items 7 
comments 37-38 

commitO method, database connection 315, 350 
Gommon Gateway Interface Scripts. See GGI Scripts 
comprehension, list 154—159, 172, 409—411, 432 
concatenation operator (+) 138 
conditional list comprehension 409—411, 432 
conditions. See if/else statement 

connection, database 
closing 314, 315 
creating 314, 315 
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connect0 method, sqlite3 315, 350 
context management protocol 120 

Controller, in MVG pattern 221 
for GAE webapps 359, 370-373 
for webapps 234-238,244-246 

“copied” sorting 144, 145-146, 172 

GREATE TABLE statement, SQL 317, 319—320 

GSS (cascading style sheet) 374-375 

GSV format, converting to Python data types 401—405 

curly braces ({ }) 

creating dictionaries 180 
creating sets 166 

cursorQ method, database connection 315, 350 
custom code 131 
custom exceptions 439 

D 

data 

for Android apps. See JSON data interchange format 

bundling with code. See classes 

duplicates in, removing 161—163, 166—167 

external. See database management system; hies 

for GAE webapps. See datastore, for GAE 

nonuniform, cleaning 148—153 

race conditions with 309-310 

Robustness Principle for 384-387 

searching for closest match 416—417 

sending to web server 275, 291 

sorting 144—147, 172 

storing. See persistence 

transforming, list comprehensions for 154—159 
database API 314-315,350 

database management system 312 
closing connection to 314,315 
commit changes to data 314, 315, 350 
connecting to 314, 315 
cursor for, manipulating data with 314, 315 
designing 316-318 
inserting data into 321, 324, 348 
integrating with Android apps 342-348 
integrating with webapps 327—341 
managing and viewing data in 326 
process for interacting with 314—315 
querying 322, 332-333 


rollback changes to data 314, 315, 350 
schema for 317 
SQLite for. See SQLite 
tables in 317, 319-320, 350 

data folder 234 

data interchange format. See JSON data interchange 
format 


data objects. See also specihc data objects 
getting next item from 54 
ID for 54 

length of, determining 32 
names of. See identihers 

datastore, for GAE 359-360, 380-383, 384-387, 395 
data types 

converting GSV data into 401-405 
converting strings to integers 54 
in datastore 360 
immutable 91, 103, 116, 138 
for JSON 285 
for list items 8,12 

date and time data 

format compatibility issues 418-42 3 
property type for 362, 384—385 

db.BlobQ type 360 
db.DatePropertyQ type 360 
db.IntegerPropertyQ type 360 
db.StringPropertyO type 360, 385 
db.TimePropertyO type 360 
db.UserPropertyQ type 360, 390 
decision statement. See if/else statement 
decorators, function 439 
def keyword 29, 191, 212 

dialogCreateAlertQ method, Android API 274, 276, 280 

dialogGetlnputQ method, Android API 295, 304-306 

dialogGetResponse() method, Android API 274, 276, 
278, 280 

dialogGetSelectedltemsQ method, Android API 278, 280 

dialogSetltemsQ method, Android API 279, 280 

dialogSetNegativeButtonTextO method, Android API 
274, 276 

dialogSetPositiveButtonTextQ method, Android API 274, 
276, 280 
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dialogSetSingleChoiceItems() method, Android API 274, 
276 

dialogShow0 method, Android API 276, 280 
dictQ factory function 180, 212 

dictionaries 178-182, 212 
accessing items in 180, 212 
compared to lists 179 
converting class data to 285-286 
creating 180, 182, 186 
dictionaries within 407—409 
keysfor 178, 180, 212 
populating 180, 212 
reading GSV data into 403-404 
values of 178,180,212 

dirQ command 225 

directory structure. See folder structure 

dist folder 42 

distribution 

creating 40—42 
updating 60—61, 65 
uploading to PyPI 48 

Dive Into Python 3 (GreateSpace) 445 
djangoforms.ModelForm class 368 
Django Project 

Form Validation Framework 368—369, 395 
templates 363-366, 395 

doctest framework 438 

documentation for Python 3 3, 80, 103 

dot notation 10, 194, 196 

double quotes. See quotes 

dumpQ function, pickle 133-134,138 

dumpsQ function, json 269, 272, 281, 291 

dynamic content 216, 217 

E 

Eclipse editor 35 

editors 35, 436. See also IDLE 

elif keyword 108. See also if/else statement 

else keyword. See if/else statement 

emacs editor 35, 436 


enableQ function, cgitb 248, 253 
end_formQ function, yate 231, 233 
entities, in datastore 360, 395 
enumerate 0 built-in function 54 
environ dictionary 300, 350 

equal sign (=) 

assignment operator 7 
in function argument dehnition 63 

errors. See exception handling; troubleshooting 

exception handling 88-95, 103. See also troubleshooting 
benehts of 95, 100 
closing hies after 114-115, 120-123 
custom exceptions 439 

dehning with try/except statement 89, 91—94 

ignoring found errors 93—94 

IndexError exception 17 

IOError exception 103, 112—114, 117—119 

for missing hies 96—98 

NameError exception 44, 118 

PickleError exception 133—134 

specihc errors, checking for 101—102 

specihc errors, details about 117-119 

TypeError exception 56-57, 116,247-249,283-285 

ValueError exception 78-79,81-82, 103 

exception objects 119,138 

except keyword. See try/except statement 

executeQ method, cursor 315, 322, 324, 350 

extend0 method, lists 10 

F 

F5key, IDLE 39,44,49,71 

factory funetions 166 

favicon.ico hle, for webapp 234 

fetchallO method, cursor 322 

fetchmanyO method, cursor 322 

fetchoneO method, cursor 322 

FieldStorage0 method, egi 244, 253, 296, 300, 350 

hies. See also persistence 

access modes for 110, 133 
appending data to 110 
checking for existence of 118 
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closing 75, 110 

closing after exception 114-115, 120-123 
GSV format, converting to Python data types 401 — 
405 

exceptions involving, determining type of 117—119 
flushing 110 

missing, exception handling for 96—98 

opening 75, 109-110 

opening in binary access mode 133 

reading data from 75-78,142-143 

rewinding 76 

splitting lines in 77-78 

writing 110—113 

writing, custom formats for 126—130 
writing, default format for 124—125 
writing, pickle library for. See pickle library 

hnally keyword 115, 138 

findO method, strings 84-86, 103 

Firefox, SQLite Manager for 326 

folder structure 

for distribution 42 
forGAE 356,370 
forwebapps 234 

for loop 15-17,32 

compared to list comprehension 432 
nesting 19-22 

forms, HTML 295 

creating from template 296—299 
Form Validation Framework for 368—369 
input restrictions for 376-377, 384-387 
sending data to GGI Scripts 300-303 
stylesheets for 374-375 

Form Validation Framework 368—369, 395 
405 Error, from web server 378 
404 Error, from web server 242 

Friedl, Jeff (author, Mastering Regular Expressions) 440 

from statement 46, 49 

functional programming concepts 157 

function decorators 439 

functions 

adding arguments to 52, 66-68 
anonymous 439 

built-in. See built-in functions (BIFs) 


chaining 146, 172 
creating 28-30, 170-171 
optional arguments for 63-64, 134 
recursive 31 
sharing. See modules 

G 

GAE (Google App Engine) 354 

configuration and setup for 356-357 

controller code for 370-373 

data modeling with 360-362 

datastore for 359, 380-383, 384-387, 395 

deploying webapps to Google cloud 391 

folder structure for 356, 370 

form generation for 368—369 

form input restrictions for 376-377, 384-387 

form stylesheets for 374—375 

MVG pattern used by 359 

SDK for, downloading 355 

troubleshooting 378 

user authorization for 389—393 

view for, desigining 363—369 

GAE Launcher 357, 391, 395 
garbage collection 116 
generators 439 

get0 method, GAE 370, 379, 395 
GET web request 370 
GIL (Global Interpreter Lock) 444 
glob module 237, 253 
Google App Engine. See GAE 
Google BigTable technology 354, 359 
GQL (Google Query Language) API 359 
green text in IDLE 4 

GUI (graphical user interface), building 443 

H 

hashes. See dictionaries 
headerQ function, yate 231, 233 

Head First HTML with GSS & XHTML (0’Reilly) 374 
Head First Programming (0’Reilly) 443 
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Head First SQL (CTReilly) 313 
helpO built-in function 80, 103 
HTML 

generating for webapp interface 230-231 
learning 226 

templates for, with Django 363-366 
HTML forms. See forms, HTML 
HTTP server 235 
http.server module 235, 253 

i 

id0 built-in function 54 
IDE 436. See also IDLE 
identifiers 7, 17, 32 

IDLE 3-5, 32 

colored syntax used in 4 
indenting enforced in 4 
preferences, setting 5 
prompt in (>>>) 4 

recalling and editing code statements 5 
running or loading code in 39, 44, 49, 71 
TAB completion 5 

if/else statement 20, 32 
elif keyword 108 
in list comprehension 432 
negating conditions in 86, 103 

images folder 234 

immutable data types 138 
lists 91, 103, 116 
numbers 116 
strings 116 

import statement 43, 46, 49, 71 
include_footerQ function, yate 230, 232, 233 
include_headerQ function, yate 230, 232 

indentation rules 

enforced in IDLE 4 
for for loops 16 
for function definitions 29 
for if statement 20 


IndexError exception 17 
index.html file, for webapp 234 
inherited classes 204-209,212 

_init_0 method 191, 212 

inoperator 16, 118, 138 
“in-place” sorting 144, 145, 172 
input 

from Android apps 278-282, 295, 304-307 

HTML forms for. See forms, HTML 

from keyboard after screen prompt 413—414, 432 

input0 built-in function 413—414, 432 

insert0 method, lists 10,14 

INSERT statement, SQL 321, 324, 348 

instances of classes 190, 191, 194, 195-196 

int0 built-in function 54 

integers, converting strings to 54 

interface. See View, in MVG pattern 

IOError exception 103, 112—114, 117—119 

I/O (input/output), handling. See files 

isinstanee0 built-in function 20-22, 32 

iterations 

forloop 15-17,19-22,32 
generating with range0 function 54—56 
while loop 16—17 

J 

JSON data interchange format 266-267, 291 
API for, using 269-272 
browser differences with 272 
data types supported by 285 
incompatibility with pickle data objects 284-285 

K 

KDevelop IDE 436 

keys, in dictionary 178, 180, 212 

keywords, displayed as orange text in IDLE 4 
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L 

lambda expressiori 439 
Learning Python (0’Reilly) 445 
len0 built-in function 10, 32 
lib folder 42 
Linux 

code editors for 35 

GAE log messages on 378 

IDEs for 436 

installing Python 3 on 3 

running GGI Scripts on 239, 272 

running GAE Launcher on 357 

transferring hies to Android device 288 

listQ built-in function 54 

list comprehension 154—159, 172, 409—411, 432 

lists 32. See also data objects 
adding items to 10-14 
bounds checking for 17 
classes inherited from 204-208 
compared to dictionaries 179 
compared to sets 167 
creating 6-7, 54 
data types in 8, 12 
duplicates in, removing 161-163 
extracting specihc item from 175—176 
getting next item from 54 
identihers for 7 
immutable 91, 103, 116 
iterating 15-17, 157 
length of, determining 10, 32 
methods for 10 
nested, checking for 20-22 
nested, creating 18-19 
nested, handling 23-25, 28-31 
numbered, creating 54 
reading GSV data into 403-404 
removing items from 10 
similarity to arrays 9—10, 17 
slice of 160, 172 

loadQ function, pickle 133,138 
loadsQ function, json 269, 276, 280, 291 


localsQ built-in function 118,138 
loops. See iterations 

M 

Mac OS X 

code editors for 35 

GAE log messages on 378 

IDEs for 436 

installing Python 3 on 3 

running GGI Scripts on 239, 272 

running GAE Launcher on 357 

transferring hies to Android device 288 

_main_namespace 45 

MANIFEST hle 42 
mappings. See dictionaries 
Mastering Regular Expressions (0’Reilly) 440 
metaclasses 439 

methods 190. See also specihc methods 
chaining 142, 172 
for classes 195-196,198-200 
creating 212 

results of, as attributes 250, 253 
self argument of 212 

ModelForm class, djangoforms 368 

Model, in MVG pattern 221 

for GAE webapps 359, 360-362 
for webapps 222-225 

Model-View-Gontroller pattern. See MVG pattern 

modules 34—36, 71 

adding functionality to 50—52 
classes in 209,212 
creating 35 

distribution for, creating 40-42 
distribution for, updating 60-61, 65 
distribution for, uploading to PyPI 48 
importing 43—44, 46 
loading in IDLE 39, 49, 71 
locations for 38, 49 
namespaces for 45-46,71 
in Python Standard Library 36 
third-party 36 
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Monty Python 17 
multiple inheritance 209 

MVG (Model-View-Gontroller) pattern 221, 232, 253, 
359 

Controller 234-238, 244-246, 370-373 
Model 222-225, 360-362 
View 226-233, 363-369 

N 

NameError exception 44, 118 
names. See identifiers 
namespaces 45-46, 71 
next0 built-in function 54 
NoSQL 359,442 
NotePad editor 35 
not in operator 161—162 
not keyword 86, 103 
numbered lists 54 

o 

object relational mapper. See ORM (object relational 
mapper) 

objects. See data objects 

open0 built-in function 75, 103, 109-110 

orange text in IDLE 4 

ORM (object relational mapper) 442 

os module 76, 300, 350 

p 

paraQ function, yate 231, 233 

parentheses ((...)) 

enclosing function arguments 29 
enclosing immutable lists 91 

pass statement 93, 103 

persistence 105 

pickle library for 132—137 
reading data from hies 222-224 
writing data to hies 110-113, 222-224 

PickleError exception 133—134 


pickle library 132-137,138 

data modeling using 222-224 
incompatibility with JSON data types 284-285 
transferring data to a database 321-325 

plus sign (+) addition or concatenation operator 138 
popO method, lists 10,175-176 
post() method, GAE 379-383, 395 
POST web request 379 

pound sign (#) preceding one-line comments 38 

print() built-in function 10, 32, 124-125 
disabling automatic new-line for 56, 71 
displaying TAB character with 56 
writing to a hle 110, 128, 138 

Programming in Python 3 (Addison-Wesley Professional) 
445 

properties, in datastore 360, 395 
@property decorator 250, 253, 285 
purple text in IDLE 4 
putQ method, GAE 395 
.pyc hle extension 42, 49 
.py hle extension 35 

PyPI (Python Package Index) 36 
registering on website 47 
uploading distributions to 48 
uploading modules to 209 

Python 2 

compared to Python 3 17 

raw_inputQ built-in function 432 

running on Android smartphones 258—259, 291 

using with Google App Engine 355 

Python 3 

compared to Python 2 17 
documentation for 3,80, 103 
editors for 35, 436 
installing 3 

interpreter for. See IDLE 
learning 445 

python3 command 

building a distribution 41 
checking for Python version 3 
installing a distribution 41 
uploading a new distribution 68 
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Python Essential Reference (Addison-Wesley Professional) 
445 

Python for Unix and Linux System Administration 
(0’Reilly) 445 

Python, Monty 17 

Python Package Index. See PyPI 

Python Standard Library 36 



querying a database 322, 332-333 

question mark (?) parameter placeholder 321, 350 

quotes or 2.2) enclosing each list item 7 

quotes, triple or enclosing comments 37 


R 

“r” access mode 110 
race conditions 309-310 
radio_buttonQ function, yate 231, 233 
rangeQ built-in function 54-56, 71 
raw_input() built-in function 432 
readlineQ method, hies 76, 103, 142 
recursion 31 

regular brackets. See parentheses 

regular expressions 440 

re module 440 

removeQ method, lists 10 

renderQ function, template 364, 366 

Robustness Principle 384-387 

rollbackQ method, database connection 315, 350 

runtime errors 88. See also exception handling; trouble- 
shooting 



schema, database 317 
scoping of variables 437 
Scripting Layer for Android. See SL4A 
scripts. See GGI scripts; SL4A 


sdist command 41 

seekQ method, hies 76,103 

SELEGT/OPTION, HTML tag 376 

SELEGT statement, SQL 322, 332-333 

self argument 192-193, 212 

self.request object 379,395 

self.response object 372, 379, 395 

semicolon (;) separating statements on one line 38 

setQ built-in function 166, 172 

sets 166, 167, 172 

setupQ built-in function 40 

setup.py file 40, 71 

single quotes. See quotes 

SL4A (Scripting Layer for Android) 258, 291 
adding Python to 263 
Android apps, creating 274-277 
automatic rotation mode, setting 264 
documentation for 274 
installing 262 

Python versions supported 258-259 
slice of a list 160, 172 
smartphones, apps on. See Android apps 
sortedO built-in function 144-147, 153, 158, 172 
sort() method, lists 144-145, 153, 172 
split() method, strings 77-78, 80-81, 103, 142 
SQL Alchemy 442 

SQLite 313,350 

closing connection to 314,315 

committing data to 314,315 

connecting to 314, 315 

cursor for, manipulating data with 314, 315 

designing database 316-318 

inserting data into 321, 324, 348 

integrating with Android apps 342-348 

integrating with webapps 327—341 

managing data in 326 

process for interacting with 314—315 

querying 322, 332—333 

rollback changes to data 314 

schema for database 317 

tables in, creating 319-320 
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sqlite3 command 326 

sqlite3 library 313, 315, 350 

SQLite Manager, for Firefox 326 

SQL (Structured Query Language) 313, 350. See 
also NoSQL; SQLite; ORM 

square brackets ([...]) 

accessing dictionary items 180, 212 
accessing specific list items 9, 18 
enclosing all list items 7,18 

Standard error (sys.stderr) 248, 291 

Standard input (sys.stdin) 291 

Standard Library, Python 36 

Standard output (sys.stdout) 126—128, 291 

start_form0 function, yate 231, 233 

start_response() function, yate 230, 232 

static content 216, 217 

static folder 370 

strQ built-in function 119, 138 

strings 

concatenating 138 
converting other objects to 119 
converting to integers 54 
displayed as green text in IDLE 4 
finding substrings in 84—86 
immutable 116 
sorting 148 
splitting 77-78, 80-81 
substitution templates for 230, 253 

stripQ method, strings 108, 138, 142 

Structured Query Language. See SQL 

stylesheets for HTML forms 374—375 

suite 16, 29, 32 

sys module 291 

sys.stdout file 126—128,138 


T 

TAB character, printing 56 

TAB completion, IDLE 5 

tables, database 317, 319-320, 350 

target identifiers, from splitQ method 77, 91 

.tar.gz file extension 42 

Template class 230, 253 

template module 364 

templates folder 234,370 

templates for GAE 363-366, 395 

testing code 438 

TextMate editor 35, 436 

third-party modules 36 

threads 444 

time data 

format compatibility issues 418-42 3 
property type for 362, 384—385 

time module 419, 432 

Tk Interface (tkinter) 443 

traceback 88, 103. See also exception handling; trouble- 
shooting 

tracing code 58-59 

triple chevron (>>>) IDLE prompt 4 

triple quotes or ‘A..’”) enclosing comments 37 

troubleshooting. See also exception handling 

404 Error, from web server 242 

405 Error, from web server 378 
Android apps 277 

GAE webapps 378 
testing code 438 
tracing code 58-59 

try/except statement 89,93—94, 101—102, 103 
finally keyword for 115,138 
with statement and 120-123 

tuples 91, 103, 116 

TypeError exception 56-57, 116,247-249,283-285 
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TJ 

u_list0 function, yate 231, 233 
unittest module 438 
urlencodeQ funcdon, urllib 291 
urllib2 module 291 
urllib module 291 
urlopenQ function, urllib2 291 
user authorization 389—393 
user input. See forms, HTML; input 
UserPropertyQ type, db 390 

v 

ValueError exception 78-79,81-82, 103 
values, part of dictionary 178, 180, 212 
variables, scope of 437 

vi editor 35, 436 

View, in MVG pattern 221 

for GAE webapps 359, 363—369 
for webapps 226-233 

vim editor 436 

w 

“w” access mode 110 
“w+” access mode 110 
“wb” access mode 133 


webapps 215-217, 253 

controlling code for 221, 234-238, 244-246 

data modeling for 221, 222-225 

designing with MVG 221 

design requirements for 218-220 

directory structure for 234 

Google App Engine for. See GAE 

input data, sending to GGI Scripts 300-303 

input forms for. See forms, HTML 

SQLite used with 327—341 

view for 221,226-233 

Web-based applications. See webapps 
web frameworks 441. See also GGI; WSGI 
web request 216, 253, 395 
web response 216-217, 253, 395 
web server 216-217, 235 

Web Server Gateway Interface (WSGI) 356, 370. See 
also GGI scripts 

while loop 16-17, 55 

WingIDE editor 35 

WingWare Python IDE 436 

with statement 120—123,138 

WSGI (Web Server Gateway Interface) 356, 370. See 
also GGI scripts 

Y 

yate (Yet Another Template Engine) library 226—233 
yield expression 439 
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